2020-11-20T13:45:50.951 INFO:root:teuthology version: 1.0.0-357fd4a 2020-11-20T13:45:50.952 DEBUG:teuthology.run:Teuthology command: teuthology --name teuthology-2020-11-16_04:10:02-fs-octopus-distro-basic-smithi --archive /home/teuthworker/archive/teuthology-2020-11-16_04:10:02-fs-octopus-distro-basic-smithi/5628507 --description fs/basic_functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} mount/fuse objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} supported-random-distros$/{centos_latest} tasks/cap-flush} --owner scheduled_teuthology@teuthology --verbose --lock --block -- /tmp/teuthology-worker.18xf_mcn.tmp 2020-11-20T13:45:50.998 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-11-20T13:45:51.057 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2020-11-16_04:10:02-fs-octopus-distro-basic-smithi/5628507 branch: octopus description: fs/basic_functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} mount/fuse objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} supported-random-distros$/{centos_latest} tasks/cap-flush} email: ceph-qa@ceph.io first_in_suite: false job_id: '5628507' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: teuthology-2020-11-16_04:10:02-fs-octopus-distro-basic-smithi nuke-on-error: true openstack: - volumes: count: 4 size: 30 - machine: disk: 200 os_type: centos os_version: '8.1' overrides: admin_socket: branch: octopus ceph: cephfs_ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 15m rados osd op timeout: 15m mds: debug mds: 20 debug ms: 1 mds bal frag: true mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 15m rados osd op timeout: 15m mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug filestore: 20 debug journal: 20 debug ms: 20 debug osd: 25 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - Replacing daemon mds.a log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: a7b62340ad27b26642bb54ec916a97c7e1f07a34 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: sha1: a7b62340ad27b26642bb54ec916a97c7e1f07a34 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: octopus sha1: a7b62340ad27b26642bb54ec916a97c7e1f07a34 owner: scheduled_teuthology@teuthology priority: 1000 repo: git://git.ceph.com/ceph.git roles: - - mon.a - mgr.y - mds.a - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.1 - - mon.b - mon.c - mgr.x - mds.b - osd.4 - osd.5 - osd.6 - osd.7 - client.2 - client.3 sha1: a7b62340ad27b26642bb54ec916a97c7e1f07a34 sleep_before_teardown: 0 suite: fs suite_branch: octopus suite_path: /home/teuthworker/src/git.ceph.com_ceph_octopus/qa suite_relpath: qa suite_repo: git://git.ceph.com/ceph.git suite_sha1: a7b62340ad27b26642bb54ec916a97c7e1f07a34 tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell rpm: - python3-cephfs extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_cap_flush teuthology_branch: master timestamp: 2020-11-16_04:10:02 tube: smithi user: teuthology verbose: true worker_log: /home/teuthworker/archive/worker_logs/worker.smithi.3621 2020-11-20T13:45:51.057 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_octopus/qa; will attempt to use it 2020-11-20T13:45:51.058 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks 2020-11-20T13:45:51.058 INFO:teuthology.run_tasks:Running task internal.check_packages... 2020-11-20T13:45:51.060 INFO:teuthology.task.internal:Checking packages... 2020-11-20T13:45:51.082 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'basic' and ceph hash 'a7b62340ad27b26642bb54ec916a97c7e1f07a34' 2020-11-20T13:45:51.082 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2020-11-20T13:45:51.083 INFO:teuthology.packaging:ref: None 2020-11-20T13:45:51.083 INFO:teuthology.packaging:tag: None 2020-11-20T13:45:51.083 INFO:teuthology.packaging:branch: octopus 2020-11-20T13:45:51.083 INFO:teuthology.packaging:sha1: a7b62340ad27b26642bb54ec916a97c7e1f07a34 2020-11-20T13:45:51.084 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=octopus 2020-11-20T13:45:51.350 INFO:teuthology.task.internal:Found packages for ceph version 15.2.6-589.g08106742 2020-11-20T13:45:51.350 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2020-11-20T13:45:51.369 INFO:teuthology.task.internal:no buildpackages task found 2020-11-20T13:45:51.369 INFO:teuthology.run_tasks:Running task internal.lock_machines... 2020-11-20T13:45:51.444 INFO:teuthology.task.internal.lock_machines:Locking machines... 2020-11-20T13:45:51.445 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-11-20T13:45:51.510 DEBUG:teuthology.lock.ops:lock_many request: {'locked_by': 'scheduled_teuthology@teuthology', 'count': 2, 'machine_type': 'smithi', 'description': '/home/teuthworker/archive/teuthology-2020-11-16_04:10:02-fs-octopus-distro-basic-smithi/5628507'} 2020-11-20T13:45:51.535 ERROR:teuthology.lock.ops:Insufficient nodes available to lock 2 smithi nodes. 2020-11-20T13:45:51.535 ERROR:teuthology.lock.ops:{"message": "only 1 nodes available"} 2020-11-20T13:45:51.536 INFO:teuthology.task.internal.lock_machines:0 smithi machines locked this try, 0/2 locked so far 2020-11-20T13:45:51.536 INFO:teuthology.task.internal.lock_machines:0 machines locked (0 new); need 2 more 2020-11-20T13:45:51.536 WARNING:teuthology.task.internal.lock_machines:Could not lock enough machines, waiting... 2020-11-20T13:46:01.553 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-11-20T13:46:11.571 DEBUG:teuthology.lock.ops:lock_many request: {'locked_by': 'scheduled_teuthology@teuthology', 'count': 2, 'machine_type': 'smithi', 'description': '/home/teuthworker/archive/teuthology-2020-11-16_04:10:02-fs-octopus-distro-basic-smithi/5628507'} 2020-11-20T13:46:11.593 DEBUG:teuthology.lock.ops:locked ubuntu@smithi040.front.sepia.ceph.com, ubuntu@smithi116.front.sepia.ceph.com 2020-11-20T13:46:11.642 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7feb22634a60>, signals=[15]) 2020-11-20T13:46:11.879 INFO:teuthology.provision.fog.smithi040:Scheduling deploy of centos 8.1 2020-11-20T13:46:11.888 INFO:teuthology.provision.fog.smithi116:Scheduling deploy of centos 8.1 2020-11-20T13:46:13.173 INFO:teuthology.orchestra.console:Power off smithi040 2020-11-20T13:46:13.174 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-11-20T13:46:13.193 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-20T13:46:13.313 INFO:teuthology.orchestra.console:Power off for smithi040 completed 2020-11-20T13:46:13.370 INFO:teuthology.orchestra.console:Power off smithi116 2020-11-20T13:46:13.371 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi116.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-11-20T13:46:13.392 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi116.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-20T13:46:13.414 INFO:teuthology.orchestra.console:Power on smithi040 2020-11-20T13:46:13.415 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-11-20T13:46:13.433 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-20T13:46:13.511 INFO:teuthology.orchestra.console:Power off for smithi116 completed 2020-11-20T13:46:13.612 INFO:teuthology.orchestra.console:Power on smithi116 2020-11-20T13:46:13.612 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi116.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-11-20T13:46:13.630 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi116.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-20T13:46:17.455 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-20T13:46:17.653 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi116.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-20T13:46:21.567 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-20T13:46:21.762 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi116.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-20T13:46:21.777 INFO:teuthology.orchestra.console:Power on for smithi040 completed 2020-11-20T13:46:21.879 INFO:teuthology.provision.fog.smithi040:Waiting for deploy to finish 2020-11-20T13:46:22.062 INFO:teuthology.orchestra.console:Power on for smithi116 completed 2020-11-20T13:46:22.163 INFO:teuthology.provision.fog.smithi116:Waiting for deploy to finish 2020-11-20T13:49:12.587 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T13:49:21.639 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T13:49:28.360 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T13:49:30.695 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T13:49:37.417 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T13:49:39.751 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T13:49:46.472 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T13:49:48.807 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T13:49:55.528 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T13:49:57.863 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T13:50:04.586 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T13:50:13.642 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T13:50:22.698 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T13:50:54.444 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-11-20T13:50:54.445 INFO:teuthology.orchestra.run.smithi116:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-11-20T13:51:03.867 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T13:51:04.199 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-11-20T13:51:04.199 INFO:teuthology.orchestra.run.smithi040:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-11-20T13:51:04.503 INFO:teuthology.provision.fog.smithi040:Node is ready 2020-11-20T13:51:04.503 INFO:teuthology.orchestra.run.smithi040:> hostname 2020-11-20T13:51:04.518 INFO:teuthology.orchestra.run.smithi040.stdout:smithi040.front.sepia.ceph.com 2020-11-20T13:51:04.518 INFO:teuthology.orchestra.run.smithi040:> grep smithi040.front.sepia.ceph.com /etc/hosts 2020-11-20T13:51:04.573 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40 smithi040.front.sepia.ceph.com smithi040 2020-11-20T13:51:04.574 INFO:teuthology.orchestra.run.smithi040:> sudo hostname smithi040 2020-11-20T13:51:04.685 INFO:teuthology.orchestra.run.smithi040:> sudo sed -i -e 's/smithi040.front.sepia.ceph.com/smithi040/g' /etc/hosts 2020-11-20T13:51:04.717 INFO:teuthology.orchestra.run.smithi040:> sudo sed -i -e 's/172.21.15.40/172.21.15.40/g' /etc/hosts 2020-11-20T13:51:04.790 INFO:teuthology.orchestra.run.smithi040:> sudo sed -i -e 's/smithi040.front.sepia.ceph.com/smithi040/g' /etc/hostname 2020-11-20T13:51:04.861 INFO:teuthology.orchestra.run.smithi040:> sudo hostname smithi040 2020-11-20T13:51:04.931 INFO:teuthology.provision.fog.smithi040:Deploy complete! 2020-11-20T13:51:04.932 DEBUG:teuthology.parallel:result is None 2020-11-20T13:51:14.713 INFO:teuthology.provision.fog.smithi116:Node is ready 2020-11-20T13:51:14.713 INFO:teuthology.orchestra.run.smithi116:> hostname 2020-11-20T13:51:14.732 INFO:teuthology.orchestra.run.smithi116.stdout:smithi116.front.sepia.ceph.com 2020-11-20T13:51:14.732 INFO:teuthology.orchestra.run.smithi116:> grep smithi116.front.sepia.ceph.com /etc/hosts 2020-11-20T13:51:14.789 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.116 smithi116.front.sepia.ceph.com smithi116 2020-11-20T13:51:14.789 INFO:teuthology.orchestra.run.smithi116:> sudo hostname smithi116 2020-11-20T13:51:14.939 INFO:teuthology.orchestra.run.smithi116:> sudo sed -i -e 's/smithi116.front.sepia.ceph.com/smithi116/g' /etc/hosts 2020-11-20T13:51:14.969 INFO:teuthology.orchestra.run.smithi116:> sudo sed -i -e 's/172.21.15.116/172.21.15.116/g' /etc/hosts 2020-11-20T13:51:15.038 INFO:teuthology.orchestra.run.smithi116:> sudo sed -i -e 's/smithi116.front.sepia.ceph.com/smithi116/g' /etc/hostname 2020-11-20T13:51:15.106 INFO:teuthology.orchestra.run.smithi116:> sudo hostname smithi116 2020-11-20T13:51:15.173 INFO:teuthology.provision.fog.smithi116:Deploy complete! 2020-11-20T13:51:15.174 DEBUG:teuthology.parallel:result is None 2020-11-20T13:51:15.289 INFO:teuthology.lock.ops:Checking smithi040.front.sepia.ceph.com 2020-11-20T13:51:15.289 INFO:teuthology.lock.ops:New key found. Updating... 2020-11-20T13:51:15.304 INFO:teuthology.lock.ops:Checking smithi116.front.sepia.ceph.com 2020-11-20T13:51:15.305 INFO:teuthology.lock.ops:New key found. Updating... 2020-11-20T13:51:15.319 INFO:teuthology.lock.ops:Updating [smithi040.front.sepia.ceph.com]: set os type and version on server 2020-11-20T13:51:15.320 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-20T13:51:15.321 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T13:51:15.388 INFO:teuthology.orchestra.run.smithi040:> true 2020-11-20T13:51:15.479 INFO:teuthology.orchestra.run.smithi040:> uname -m 2020-11-20T13:51:15.534 INFO:teuthology.orchestra.run.smithi040.stdout:x86_64 2020-11-20T13:51:15.535 INFO:teuthology.orchestra.run.smithi040:> cat /etc/os-release 2020-11-20T13:51:15.590 INFO:teuthology.orchestra.run.smithi040.stdout:NAME="CentOS Linux" 2020-11-20T13:51:15.590 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION="8 (Core)" 2020-11-20T13:51:15.590 INFO:teuthology.orchestra.run.smithi040.stdout:ID="centos" 2020-11-20T13:51:15.591 INFO:teuthology.orchestra.run.smithi040.stdout:ID_LIKE="rhel fedora" 2020-11-20T13:51:15.591 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION_ID="8" 2020-11-20T13:51:15.591 INFO:teuthology.orchestra.run.smithi040.stdout:PLATFORM_ID="platform:el8" 2020-11-20T13:51:15.591 INFO:teuthology.orchestra.run.smithi040.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2020-11-20T13:51:15.591 INFO:teuthology.orchestra.run.smithi040.stdout:ANSI_COLOR="0;31" 2020-11-20T13:51:15.592 INFO:teuthology.orchestra.run.smithi040.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2020-11-20T13:51:15.592 INFO:teuthology.orchestra.run.smithi040.stdout:HOME_URL="https://www.centos.org/" 2020-11-20T13:51:15.592 INFO:teuthology.orchestra.run.smithi040.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-11-20T13:51:15.592 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:51:15.592 INFO:teuthology.orchestra.run.smithi040.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2020-11-20T13:51:15.593 INFO:teuthology.orchestra.run.smithi040.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2020-11-20T13:51:15.593 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-11-20T13:51:15.593 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2020-11-20T13:51:15.593 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:51:15.594 INFO:teuthology.lock.ops:Updating smithi040.front.sepia.ceph.com on lock server 2020-11-20T13:51:15.610 INFO:teuthology.lock.ops:Updating [smithi116.front.sepia.ceph.com]: set os type and version on server 2020-11-20T13:51:15.610 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-20T13:51:15.610 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T13:51:15.677 INFO:teuthology.orchestra.run.smithi116:> true 2020-11-20T13:51:15.834 INFO:teuthology.orchestra.run.smithi116:> uname -m 2020-11-20T13:51:15.888 INFO:teuthology.orchestra.run.smithi116.stdout:x86_64 2020-11-20T13:51:15.888 INFO:teuthology.orchestra.run.smithi116:> cat /etc/os-release 2020-11-20T13:51:15.941 INFO:teuthology.orchestra.run.smithi116.stdout:NAME="CentOS Linux" 2020-11-20T13:51:15.941 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION="8 (Core)" 2020-11-20T13:51:15.941 INFO:teuthology.orchestra.run.smithi116.stdout:ID="centos" 2020-11-20T13:51:15.941 INFO:teuthology.orchestra.run.smithi116.stdout:ID_LIKE="rhel fedora" 2020-11-20T13:51:15.942 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION_ID="8" 2020-11-20T13:51:15.942 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM_ID="platform:el8" 2020-11-20T13:51:15.942 INFO:teuthology.orchestra.run.smithi116.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2020-11-20T13:51:15.942 INFO:teuthology.orchestra.run.smithi116.stdout:ANSI_COLOR="0;31" 2020-11-20T13:51:15.942 INFO:teuthology.orchestra.run.smithi116.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2020-11-20T13:51:15.943 INFO:teuthology.orchestra.run.smithi116.stdout:HOME_URL="https://www.centos.org/" 2020-11-20T13:51:15.943 INFO:teuthology.orchestra.run.smithi116.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-11-20T13:51:15.943 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T13:51:15.943 INFO:teuthology.orchestra.run.smithi116.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2020-11-20T13:51:15.943 INFO:teuthology.orchestra.run.smithi116.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2020-11-20T13:51:15.943 INFO:teuthology.orchestra.run.smithi116.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-11-20T13:51:15.944 INFO:teuthology.orchestra.run.smithi116.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2020-11-20T13:51:15.944 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T13:51:15.944 INFO:teuthology.lock.ops:Updating smithi116.front.sepia.ceph.com on lock server 2020-11-20T13:51:15.963 INFO:teuthology.task.internal.lock_machines:2 smithi machines locked this try, 2/2 locked so far 2020-11-20T13:51:15.997 INFO:teuthology.task.internal.lock_machines:Locked targets: smithi040.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCc+NNCop57dEEIxzisCShx+4lQwqNnkwsklSPNn3QMj4SM0iuHLzR9gJES8K+idQbyVjnJdxNXsW9oqjMIGkAc61AWlcbAGZbhHLA93FICp9JjaONyJ10GHTiIzlCieksFsIJrzHEuBRoTOxzfmAoHlkM4QKp03eovsiornj4Y86di2Puk4H68QFw1q5HzZOJajU9jw7FNGABe2+2YNdd6Y/wAwh3rmpY43B4o1FrrLow6bpia+5Fe8/RN/fG30uEWZ0/6KP9C+QjLqnxyE4wnPx9fJXvXb3v4/oF+wfEKIsaW8/S5ufJm4x2/XxrTKSJF8DvNs3VNfZJ/ZZ0Q1/T5MJm0yv6Qqt+SO+ia6d5O2597FYiBPxLF8tVZpXWIdvYm+i8rxevgrdCN40FgYuukDuUrNEuaezpPXHKqTiQRu2vBXRfNd23F3B6wOKlpbJ+AwIOxuuSsAzRApymSMgwF22gKxqZvlflH600AUHdcuyTh2P3I7QEnHWJah5AcYts= smithi116.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCuDX2LoD2ExaYCl+93xaiz+YKTfMJuUpuOuZUYBxtXw+Gd2Fc98v2x+Q/9Wyw8sE9yrIA2lJFZFo0QDr4/s+Vdil+vYqGsvqh9/TWzKJ3BCSSjNsO6Ux6oxj1PIqwHwWm8VUMLtw4E5nBlVlGwS1OsNNs8HSMKc1BWq8+CZj5ocEzy99vjhddvLLzHZliEyQLJVtaFGnLpMnpixCeRRgydkOiv2lbDUaBoeNyV3GcD3syouvI6hR9QaZtS46HY4QHVfvWwyvLITHvnn0lQkLhwWuhjndZ+Ly+jXUCWOF+Nv4COUj3OgX8G5eExRb97jE05DE1E8N2iX31HOH3UKIXZJ4yS3v4odJWd6kT8WYY16tVAZslvhYjZbOfoyISPHLPmJCAamgWo4hd01KhTDawrwsu0rvoe7HcL/IQqCbsaA5EpqWHYGQ5dptLlqfud5+FjlYkPnUsw55YWfZPCbb91nXhK7SWqdyHbC0zUtckqp/Lb5wSJ40AvpIuE589JMQE= 2020-11-20T13:51:15.998 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-11-20T13:51:16.047 INFO:teuthology.run_tasks:Running task internal.save_config... 2020-11-20T13:51:16.055 INFO:teuthology.task.internal:Saving configuration 2020-11-20T13:51:16.072 INFO:teuthology.run_tasks:Running task internal.check_lock... 2020-11-20T13:51:16.092 INFO:teuthology.task.internal.check_lock:Checking locks... 2020-11-20T13:51:16.110 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-11-20 13:46:11.579030', 'description': '/home/teuthworker/archive/teuthology-2020-11-16_04:10:02-fs-octopus-distro-basic-smithi/5628507', 'locked_by': 'scheduled_teuthology@teuthology', 'up': True, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPz+OH7/S5mKQM3rk2Sw5D0RCoO1kO+ueVqAdoUOhm9ZRg7zgcz0RgGgyPNxu2g6JfVqwdTAk0IRqVOC/rDITcc=', 'os_version': '8', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'centos', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi040.front.sepia.ceph.com'} 2020-11-20T13:51:16.126 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-11-20 13:46:11.579051', 'description': '/home/teuthworker/archive/teuthology-2020-11-16_04:10:02-fs-octopus-distro-basic-smithi/5628507', 'locked_by': 'scheduled_teuthology@teuthology', 'up': True, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEk4g6uFp4MNSJoQW1C3JEdgWHpMLFRSHjpfwKjmNQ/MXtCrNmR3hHtBSmeex6b3H6/1aSwyFYsLrWPACPW8djA=', 'os_version': '8', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'centos', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi116.front.sepia.ceph.com'} 2020-11-20T13:51:16.126 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2020-11-20T13:51:16.136 INFO:teuthology.task.internal:roles: ubuntu@smithi040.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.1'] 2020-11-20T13:51:16.137 INFO:teuthology.task.internal:roles: ubuntu@smithi116.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.2', 'client.3'] 2020-11-20T13:51:16.137 INFO:teuthology.run_tasks:Running task console_log... 2020-11-20T13:51:16.210 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7feb2261e840>, signals=[15]) 2020-11-20T13:51:16.211 INFO:teuthology.run_tasks:Running task internal.connect... 2020-11-20T13:51:16.222 INFO:teuthology.task.internal:Opening connections... 2020-11-20T13:51:16.222 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi040.front.sepia.ceph.com 2020-11-20T13:51:16.224 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T13:51:16.297 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi116.front.sepia.ceph.com 2020-11-20T13:51:16.299 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T13:51:16.369 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2020-11-20T13:51:16.380 INFO:teuthology.orchestra.run.smithi040:> uname -m 2020-11-20T13:51:16.399 INFO:teuthology.orchestra.run.smithi040.stdout:x86_64 2020-11-20T13:51:16.400 INFO:teuthology.orchestra.run.smithi040:> cat /etc/os-release 2020-11-20T13:51:16.455 INFO:teuthology.orchestra.run.smithi040.stdout:NAME="CentOS Linux" 2020-11-20T13:51:16.455 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION="8 (Core)" 2020-11-20T13:51:16.455 INFO:teuthology.orchestra.run.smithi040.stdout:ID="centos" 2020-11-20T13:51:16.456 INFO:teuthology.orchestra.run.smithi040.stdout:ID_LIKE="rhel fedora" 2020-11-20T13:51:16.456 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION_ID="8" 2020-11-20T13:51:16.456 INFO:teuthology.orchestra.run.smithi040.stdout:PLATFORM_ID="platform:el8" 2020-11-20T13:51:16.456 INFO:teuthology.orchestra.run.smithi040.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2020-11-20T13:51:16.456 INFO:teuthology.orchestra.run.smithi040.stdout:ANSI_COLOR="0;31" 2020-11-20T13:51:16.457 INFO:teuthology.orchestra.run.smithi040.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2020-11-20T13:51:16.457 INFO:teuthology.orchestra.run.smithi040.stdout:HOME_URL="https://www.centos.org/" 2020-11-20T13:51:16.457 INFO:teuthology.orchestra.run.smithi040.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-11-20T13:51:16.457 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:51:16.458 INFO:teuthology.orchestra.run.smithi040.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2020-11-20T13:51:16.458 INFO:teuthology.orchestra.run.smithi040.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2020-11-20T13:51:16.458 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-11-20T13:51:16.458 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2020-11-20T13:51:16.458 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:51:16.459 INFO:teuthology.lock.ops:Updating smithi040.front.sepia.ceph.com on lock server 2020-11-20T13:51:16.474 INFO:teuthology.orchestra.run.smithi116:> uname -m 2020-11-20T13:51:16.489 INFO:teuthology.orchestra.run.smithi116.stdout:x86_64 2020-11-20T13:51:16.489 INFO:teuthology.orchestra.run.smithi116:> cat /etc/os-release 2020-11-20T13:51:16.542 INFO:teuthology.orchestra.run.smithi116.stdout:NAME="CentOS Linux" 2020-11-20T13:51:16.542 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION="8 (Core)" 2020-11-20T13:51:16.542 INFO:teuthology.orchestra.run.smithi116.stdout:ID="centos" 2020-11-20T13:51:16.542 INFO:teuthology.orchestra.run.smithi116.stdout:ID_LIKE="rhel fedora" 2020-11-20T13:51:16.543 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION_ID="8" 2020-11-20T13:51:16.543 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM_ID="platform:el8" 2020-11-20T13:51:16.543 INFO:teuthology.orchestra.run.smithi116.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2020-11-20T13:51:16.543 INFO:teuthology.orchestra.run.smithi116.stdout:ANSI_COLOR="0;31" 2020-11-20T13:51:16.543 INFO:teuthology.orchestra.run.smithi116.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2020-11-20T13:51:16.544 INFO:teuthology.orchestra.run.smithi116.stdout:HOME_URL="https://www.centos.org/" 2020-11-20T13:51:16.544 INFO:teuthology.orchestra.run.smithi116.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-11-20T13:51:16.544 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T13:51:16.544 INFO:teuthology.orchestra.run.smithi116.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2020-11-20T13:51:16.544 INFO:teuthology.orchestra.run.smithi116.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2020-11-20T13:51:16.545 INFO:teuthology.orchestra.run.smithi116.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-11-20T13:51:16.545 INFO:teuthology.orchestra.run.smithi116.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2020-11-20T13:51:16.545 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T13:51:16.545 INFO:teuthology.lock.ops:Updating smithi116.front.sepia.ceph.com on lock server 2020-11-20T13:51:16.561 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2020-11-20T13:51:16.583 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2020-11-20T13:51:16.593 INFO:teuthology.task.internal:Checking for old test directory... 2020-11-20T13:51:16.594 INFO:teuthology.orchestra.run.smithi040:> test '!' -e /home/ubuntu/cephtest 2020-11-20T13:51:16.596 INFO:teuthology.orchestra.run.smithi116:> test '!' -e /home/ubuntu/cephtest 2020-11-20T13:51:16.610 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2020-11-20T13:51:16.621 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2020-11-20T13:51:16.622 INFO:teuthology.orchestra.run.smithi040:> test -z $(ls -A /var/lib/ceph) 2020-11-20T13:51:16.653 INFO:teuthology.orchestra.run.smithi116:> test -z $(ls -A /var/lib/ceph) 2020-11-20T13:51:16.690 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2020-11-20T13:51:16.733 INFO:teuthology.run_tasks:Running task kernel... 2020-11-20T13:51:16.749 DEBUG:teuthology.task.kernel:normalized config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mgr.y': {'kdb': True, 'sha1': 'distro'}, 'mds.a': {'kdb': True, 'sha1': 'distro'}, 'osd.0': {'kdb': True, 'sha1': 'distro'}, 'osd.1': {'kdb': True, 'sha1': 'distro'}, 'osd.2': {'kdb': True, 'sha1': 'distro'}, 'osd.3': {'kdb': True, 'sha1': 'distro'}, 'client.0': {'kdb': True, 'sha1': 'distro'}, 'client.1': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'mon.c': {'kdb': True, 'sha1': 'distro'}, 'mgr.x': {'kdb': True, 'sha1': 'distro'}, 'mds.b': {'kdb': True, 'sha1': 'distro'}, 'osd.4': {'kdb': True, 'sha1': 'distro'}, 'osd.5': {'kdb': True, 'sha1': 'distro'}, 'osd.6': {'kdb': True, 'sha1': 'distro'}, 'osd.7': {'kdb': True, 'sha1': 'distro'}, 'client.2': {'kdb': True, 'sha1': 'distro'}, 'client.3': {'kdb': True, 'sha1': 'distro'}} 2020-11-20T13:51:16.750 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2020-11-20T13:51:16.750 INFO:teuthology.orchestra.run.smithi040:> uname -r 2020-11-20T13:51:16.767 INFO:teuthology.orchestra.run.smithi040.stdout:4.18.0-193.28.1.el8_2.x86_64 2020-11-20T13:51:16.767 INFO:teuthology.task.kernel:Running kernel on smithi040: 4.18.0-193.28.1.el8_2.x86_64 2020-11-20T13:51:16.768 INFO:teuthology.orchestra.run.smithi040:> sudo yum install -y kernel 2020-11-20T13:51:19.254 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS-8 - AppStream 40 MB/s | 7.0 MB 00:00 2020-11-20T13:51:20.568 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS-8 - Base 25 MB/s | 2.2 MB 00:00 2020-11-20T13:51:21.026 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS-8 - Extras 84 kB/s | 5.5 kB 00:00 2020-11-20T13:51:21.520 INFO:teuthology.orchestra.run.smithi040.stdout:Extra Packages for Enterprise Linux 19 MB/s | 8.3 MB 00:00 2020-11-20T13:51:23.392 INFO:teuthology.orchestra.run.smithi040.stdout:lab-extras 4.5 MB/s | 23 kB 00:00 2020-11-20T13:51:24.414 INFO:teuthology.orchestra.run.smithi040.stdout:Package kernel-4.18.0-147.el8.x86_64 is already installed. 2020-11-20T13:51:24.414 INFO:teuthology.orchestra.run.smithi040.stdout:Package kernel-4.18.0-193.28.1.el8_2.x86_64 is already installed. 2020-11-20T13:51:24.584 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2020-11-20T13:51:24.586 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2020-11-20T13:51:24.586 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2020-11-20T13:51:24.680 INFO:teuthology.orchestra.run.smithi040:> echo no | sudo yum reinstall kernel || true 2020-11-20T13:51:25.114 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:00:02 ago on Fri 20 Nov 2020 01:51:23 PM UTC. 2020-11-20T13:51:25.373 INFO:teuthology.orchestra.run.smithi040.stdout:Installed package kernel-4.18.0-147.el8.x86_64 (from anaconda) not available. 2020-11-20T13:51:25.373 INFO:teuthology.orchestra.run.smithi040.stdout:Installed package kernel-4.18.0-193.28.1.el8_2.x86_64 (from BaseOS) not available. 2020-11-20T13:51:25.432 INFO:teuthology.orchestra.run.smithi040.stderr:Error: No packages marked for reinstall. 2020-11-20T13:51:25.501 INFO:teuthology.orchestra.run.smithi040:> sudo yum reinstall -y kernel || true 2020-11-20T13:51:25.933 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:00:02 ago on Fri 20 Nov 2020 01:51:23 PM UTC. 2020-11-20T13:51:26.181 INFO:teuthology.orchestra.run.smithi040.stdout:Installed package kernel-4.18.0-147.el8.x86_64 (from anaconda) not available. 2020-11-20T13:51:26.181 INFO:teuthology.orchestra.run.smithi040.stdout:Installed package kernel-4.18.0-193.28.1.el8_2.x86_64 (from BaseOS) not available. 2020-11-20T13:51:26.212 INFO:teuthology.orchestra.run.smithi040.stderr:Error: No packages marked for reinstall. 2020-11-20T13:51:26.251 INFO:teuthology.orchestra.run.smithi040:> rpm -q kernel --last | head -n 1 2020-11-20T13:51:26.329 INFO:teuthology.orchestra.run.smithi040.stdout:kernel-4.18.0-193.28.1.el8_2.x86_64 Sun 15 Nov 2020 12:12:26 PM UTC 2020-11-20T13:51:26.330 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-193.28.1.el8_2.x86_64 2020-11-20T13:51:26.331 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2020-11-20T13:51:26.331 INFO:teuthology.orchestra.run.smithi116:> uname -r 2020-11-20T13:51:26.348 INFO:teuthology.orchestra.run.smithi116.stdout:4.18.0-193.28.1.el8_2.x86_64 2020-11-20T13:51:26.348 INFO:teuthology.task.kernel:Running kernel on smithi116: 4.18.0-193.28.1.el8_2.x86_64 2020-11-20T13:51:26.349 INFO:teuthology.orchestra.run.smithi116:> sudo yum install -y kernel 2020-11-20T13:51:28.953 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS-8 - AppStream 42 MB/s | 7.0 MB 00:00 2020-11-20T13:51:30.155 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS-8 - Base 25 MB/s | 2.2 MB 00:00 2020-11-20T13:51:30.576 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS-8 - Extras 86 kB/s | 5.5 kB 00:00 2020-11-20T13:51:31.011 INFO:teuthology.orchestra.run.smithi116.stdout:Extra Packages for Enterprise Linux 22 MB/s | 8.3 MB 00:00 2020-11-20T13:51:32.773 INFO:teuthology.orchestra.run.smithi116.stdout:lab-extras 4.6 MB/s | 23 kB 00:00 2020-11-20T13:51:33.671 INFO:teuthology.orchestra.run.smithi116.stdout:Package kernel-4.18.0-147.el8.x86_64 is already installed. 2020-11-20T13:51:33.671 INFO:teuthology.orchestra.run.smithi116.stdout:Package kernel-4.18.0-193.28.1.el8_2.x86_64 is already installed. 2020-11-20T13:51:33.833 INFO:teuthology.orchestra.run.smithi116.stdout:Dependencies resolved. 2020-11-20T13:51:33.834 INFO:teuthology.orchestra.run.smithi116.stdout:Nothing to do. 2020-11-20T13:51:33.834 INFO:teuthology.orchestra.run.smithi116.stdout:Complete! 2020-11-20T13:51:33.969 INFO:teuthology.orchestra.run.smithi116:> echo no | sudo yum reinstall kernel || true 2020-11-20T13:51:34.371 INFO:teuthology.orchestra.run.smithi116.stdout:Last metadata expiration check: 0:00:02 ago on Fri 20 Nov 2020 01:51:32 PM UTC. 2020-11-20T13:51:34.610 INFO:teuthology.orchestra.run.smithi116.stdout:Installed package kernel-4.18.0-147.el8.x86_64 (from anaconda) not available. 2020-11-20T13:51:34.610 INFO:teuthology.orchestra.run.smithi116.stdout:Installed package kernel-4.18.0-193.28.1.el8_2.x86_64 (from BaseOS) not available. 2020-11-20T13:51:34.687 INFO:teuthology.orchestra.run.smithi116.stderr:Error: No packages marked for reinstall. 2020-11-20T13:51:34.732 INFO:teuthology.orchestra.run.smithi116:> sudo yum reinstall -y kernel || true 2020-11-20T13:51:35.137 INFO:teuthology.orchestra.run.smithi116.stdout:Last metadata expiration check: 0:00:03 ago on Fri 20 Nov 2020 01:51:32 PM UTC. 2020-11-20T13:51:35.410 INFO:teuthology.orchestra.run.smithi116.stdout:Installed package kernel-4.18.0-147.el8.x86_64 (from anaconda) not available. 2020-11-20T13:51:35.410 INFO:teuthology.orchestra.run.smithi116.stdout:Installed package kernel-4.18.0-193.28.1.el8_2.x86_64 (from BaseOS) not available. 2020-11-20T13:51:35.450 INFO:teuthology.orchestra.run.smithi116.stderr:Error: No packages marked for reinstall. 2020-11-20T13:51:35.492 INFO:teuthology.orchestra.run.smithi116:> rpm -q kernel --last | head -n 1 2020-11-20T13:51:35.551 INFO:teuthology.orchestra.run.smithi116.stdout:kernel-4.18.0-193.28.1.el8_2.x86_64 Sun 15 Nov 2020 12:12:26 PM UTC 2020-11-20T13:51:35.552 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-193.28.1.el8_2.x86_64 2020-11-20T13:51:35.552 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2020-11-20T13:51:35.553 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2020-11-20T13:51:35.553 INFO:teuthology.orchestra.run.smithi040:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-11-20T13:51:35.601 INFO:teuthology.orchestra.run.smithi040.stdout:ttyS1 2020-11-20T13:51:35.615 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2020-11-20T13:51:35.615 INFO:teuthology.orchestra.run.smithi116:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-11-20T13:51:35.650 INFO:teuthology.orchestra.run.smithi116.stdout:ttyS1 2020-11-20T13:51:35.664 INFO:teuthology.run_tasks:Running task internal.base... 2020-11-20T13:51:35.679 INFO:teuthology.task.internal:Creating test directory... 2020-11-20T13:51:35.679 INFO:teuthology.orchestra.run.smithi040:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-11-20T13:51:35.682 INFO:teuthology.orchestra.run.smithi116:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-11-20T13:51:35.718 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2020-11-20T13:51:35.725 INFO:teuthology.run_tasks:Running task internal.archive... 2020-11-20T13:51:35.735 INFO:teuthology.task.internal:Creating archive directory... 2020-11-20T13:51:35.736 INFO:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-11-20T13:51:35.739 INFO:teuthology.orchestra.run.smithi116:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-11-20T13:51:35.781 INFO:teuthology.run_tasks:Running task internal.coredump... 2020-11-20T13:51:35.789 INFO:teuthology.task.internal:Enabling coredump saving... 2020-11-20T13:51:35.789 INFO:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-11-20T13:51:35.808 INFO:teuthology.orchestra.run.smithi116:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-11-20T13:51:35.842 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-20T13:51:35.861 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-20T13:51:35.869 INFO:teuthology.orchestra.run.smithi116.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-20T13:51:35.882 INFO:teuthology.orchestra.run.smithi116.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-20T13:51:35.884 INFO:teuthology.run_tasks:Running task internal.sudo... 2020-11-20T13:51:35.892 INFO:teuthology.task.internal:Configuring sudo... 2020-11-20T13:51:35.892 INFO:teuthology.orchestra.run.smithi040:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-11-20T13:51:35.905 INFO:teuthology.orchestra.run.smithi116:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-11-20T13:51:35.937 INFO:teuthology.run_tasks:Running task internal.syslog... 2020-11-20T13:51:35.949 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2020-11-20T13:51:35.950 INFO:teuthology.orchestra.run.smithi040:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-11-20T13:51:35.979 INFO:teuthology.orchestra.run.smithi116:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-11-20T13:51:35.993 INFO:teuthology.orchestra.run.smithi040:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-11-20T13:51:36.069 INFO:teuthology.orchestra.run.smithi040:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2020-11-20T13:51:36.138 INFO:teuthology.orchestra.run.smithi040:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-11-20T13:51:36.172 INFO:teuthology.orchestra.run.smithi040:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2020-11-20T13:51:36.227 INFO:teuthology.orchestra.run.smithi040:> set -ex 2020-11-20T13:51:36.227 INFO:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-20T13:51:36.281 INFO:teuthology.orchestra.run.smithi040.stderr:+ sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-20T13:51:36.297 INFO:teuthology.orchestra.run.smithi040.stderr:0+1 records in 2020-11-20T13:51:36.297 INFO:teuthology.orchestra.run.smithi040.stderr:0+1 records out 2020-11-20T13:51:36.298 INFO:teuthology.orchestra.run.smithi040.stderr:152 bytes copied, 3.1242e-05 s, 4.9 MB/s 2020-11-20T13:51:36.298 INFO:teuthology.orchestra.run.smithi116:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-11-20T13:51:36.332 INFO:teuthology.orchestra.run.smithi116:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2020-11-20T13:51:36.387 INFO:teuthology.orchestra.run.smithi116:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-11-20T13:51:36.461 INFO:teuthology.orchestra.run.smithi116:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2020-11-20T13:51:36.508 INFO:teuthology.orchestra.run.smithi116:> set -ex 2020-11-20T13:51:36.509 INFO:teuthology.orchestra.run.smithi116:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-20T13:51:36.561 INFO:teuthology.orchestra.run.smithi116.stderr:+ sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-20T13:51:36.574 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records in 2020-11-20T13:51:36.574 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records out 2020-11-20T13:51:36.575 INFO:teuthology.orchestra.run.smithi116.stderr:152 bytes copied, 2.6621e-05 s, 5.7 MB/s 2020-11-20T13:51:36.576 INFO:teuthology.orchestra.run.smithi040:> sudo service rsyslog restart 2020-11-20T13:51:36.579 INFO:teuthology.orchestra.run.smithi116:> sudo service rsyslog restart 2020-11-20T13:51:36.659 INFO:teuthology.orchestra.run.smithi040.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2020-11-20T13:51:36.683 INFO:teuthology.orchestra.run.smithi116.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2020-11-20T13:51:37.136 INFO:teuthology.run_tasks:Running task internal.timer... 2020-11-20T13:51:37.147 INFO:teuthology.task.internal:Starting timer... 2020-11-20T13:51:37.147 INFO:teuthology.run_tasks:Running task pcp... 2020-11-20T13:51:37.179 INFO:teuthology.run_tasks:Running task selinux... 2020-11-20T13:51:37.194 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2020-11-20T13:51:37.194 INFO:teuthology.orchestra.run.smithi040:> sudo service auditd rotate 2020-11-20T13:51:37.263 INFO:teuthology.orchestra.run.smithi040.stdout:Rotating logs: [ OK ] 2020-11-20T13:51:37.264 INFO:teuthology.orchestra.run.smithi116:> sudo service auditd rotate 2020-11-20T13:51:37.324 INFO:teuthology.orchestra.run.smithi116.stdout:Rotating logs: [ OK ] 2020-11-20T13:51:37.325 DEBUG:teuthology.task.selinux:Getting current SELinux state 2020-11-20T13:51:37.326 INFO:teuthology.orchestra.run.smithi040:> /usr/sbin/getenforce 2020-11-20T13:51:37.346 INFO:teuthology.orchestra.run.smithi040.stdout:Permissive 2020-11-20T13:51:37.347 INFO:teuthology.orchestra.run.smithi116:> /usr/sbin/getenforce 2020-11-20T13:51:37.368 INFO:teuthology.orchestra.run.smithi116.stdout:Permissive 2020-11-20T13:51:37.368 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi040.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi116.front.sepia.ceph.com': 'permissive'} 2020-11-20T13:51:37.369 INFO:teuthology.orchestra.run.smithi040:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|scontext=system_u:system_r:logrotate_t:s0\)' 2020-11-20T13:51:37.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-20T13:51:37.422 INFO:teuthology.orchestra.run.smithi116:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|scontext=system_u:system_r:logrotate_t:s0\)' 2020-11-20T13:51:37.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-20T13:51:37.450 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2020-11-20T13:51:37.450 INFO:teuthology.orchestra.run.smithi040:> sudo /usr/sbin/setenforce permissive 2020-11-20T13:51:37.495 INFO:teuthology.orchestra.run.smithi116:> sudo /usr/sbin/setenforce permissive 2020-11-20T13:51:37.535 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2020-11-20T13:51:37.547 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master was just updated; assuming it is current 2020-11-20T13:51:37.548 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master to branch origin/master 2020-11-20T13:51:37.559 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}] 2020-11-20T13:51:37.561 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi040.front.sepia.ceph.com,smithi116.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/cephlab.yml 2020-11-20T13:51:37.863 INFO:teuthology.task.ansible.out:Using /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/ansible.cfg as config file 2020-11-20T13:51:38.659 INFO:teuthology.task.ansible.out:[WARNING]: file /home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/roles/testnode/tasks/setup-opensuse.yml is empty and had no tasks to include 2020-11-20T13:51:41.047 INFO:teuthology.task.ansible.out: PLAY [all] ********************************************************************* 2020-11-20T13:51:41.066 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:41 +0000 (0:00:00.065) 0:00:00.065 ******* 2020-11-20T13:51:41.078 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:41 +0000 (0:00:00.011) 0:00:00.076 ******* 2020-11-20T13:51:43.485 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi040.front.sepia.ceph.com] 2020-11-20T13:51:43.548 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:43 +0000 (0:00:02.470) 0:00:02.547 ******* 2020-11-20T13:51:43.557 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] 2020-11-20T13:51:43.624 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:43 +0000 (0:00:00.075) 0:00:02.623 ******* 2020-11-20T13:51:43.857 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** 2020-11-20T13:51:43.859 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2020-11-20T13:51:43.887 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:43 +0000 (0:00:00.262) 0:00:02.886 ******* 2020-11-20T13:51:43.892 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2020-11-20T13:51:43.925 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:43 +0000 (0:00:00.037) 0:00:02.924 ******* 2020-11-20T13:51:44.270 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** 2020-11-20T13:51:44.272 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2020-11-20T13:51:44.303 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:44 +0000 (0:00:00.377) 0:00:03.302 ******* 2020-11-20T13:51:44.309 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2020-11-20T13:51:44.341 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:44 +0000 (0:00:00.037) 0:00:03.340 ******* 2020-11-20T13:51:44.345 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** 2020-11-20T13:51:44.347 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:44.378 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:44 +0000 (0:00:00.036) 0:00:03.377 ******* 2020-11-20T13:51:44.384 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:44.417 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:44 +0000 (0:00:00.039) 0:00:03.416 ******* 2020-11-20T13:51:44.680 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** 2020-11-20T13:51:44.682 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-20T13:51:44.712 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:44 +0000 (0:00:00.294) 0:00:03.711 ******* 2020-11-20T13:51:44.718 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-20T13:51:44.751 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:44 +0000 (0:00:00.038) 0:00:03.750 ******* 2020-11-20T13:51:45.498 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] *************** 2020-11-20T13:51:45.500 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1605880304.7240472-42581354798358/source", "state": "file", "uid": 0} 2020-11-20T13:51:45.529 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:45 +0000 (0:00:00.777) 0:00:04.528 ******* 2020-11-20T13:51:45.535 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1605880304.762929-119081336774844/source", "state": "file", "uid": 0} 2020-11-20T13:51:45.570 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:45 +0000 (0:00:00.040) 0:00:04.568 ******* 2020-11-20T13:51:45.906 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Add authorized keys for the ansible user.] ************* 2020-11-20T13:51:45.907 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv 2020-11-20T13:51:45.909 INFO:teuthology.task.ansible.out:4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2020-11-20T13:51:45.939 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv 2020-11-20T13:51:45.941 INFO:teuthology.task.ansible.out:4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2020-11-20T13:51:46.047 INFO:teuthology.task.ansible.out: PLAY [teuthology] ************************************************************** skipping: no hosts matched 2020-11-20T13:51:46.054 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2020-11-20T13:51:46.202 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:46 +0000 (0:00:00.632) 0:00:05.201 ******* 2020-11-20T13:51:46.232 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:46 +0000 (0:00:00.029) 0:00:05.230 ******* 2020-11-20T13:51:46.857 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* 2020-11-20T13:51:46.859 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] 2020-11-20T13:51:46.901 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] 2020-11-20T13:51:46.942 INFO:teuthology.task.ansible.out:[WARNING]: Using run_once with the free strategy is not currently supported. This task will still be executed for every host in the inventory list. 2020-11-20T13:51:46.943 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:46 +0000 (0:00:00.710) 0:00:05.940 ******* 2020-11-20T13:51:47.005 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:46 +0000 (0:00:00.062) 0:00:06.003 ******* 2020-11-20T13:51:47.370 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** 2020-11-20T13:51:47.372 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.147686", "end": "2020-11-20 13:51:47.347084", "rc": 0, "start": "2020-11-20 13:51:47.199398", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2020-11-20T13:51:47.403 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:47 +0000 (0:00:00.398) 0:00:06.402 ******* 2020-11-20T13:51:47.410 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.153651", "end": "2020-11-20 13:51:47.356653", "rc": 0, "start": "2020-11-20 13:51:47.203002", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2020-11-20T13:51:47.442 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:47 +0000 (0:00:00.038) 0:00:06.440 ******* 2020-11-20T13:51:47.615 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2020-11-20T13:51:47.617 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.009803", "end": "2020-11-20 13:51:47.597698", "failed_when_result": false, "rc": 0, "start": "2020-11-20 13:51:47.587895", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2020-11-20T13:51:47.649 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:47 +0000 (0:00:00.206) 0:00:06.647 ******* 2020-11-20T13:51:47.654 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.011312", "end": "2020-11-20 13:51:47.618389", "failed_when_result": false, "rc": 0, "start": "2020-11-20 13:51:47.607077", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2020-11-20T13:51:47.687 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:47 +0000 (0:00:00.038) 0:00:06.685 ******* 2020-11-20T13:51:47.691 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2020-11-20T13:51:47.693 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"ansible_facts": {"perform_users_role": false}, "changed": false} 2020-11-20T13:51:47.725 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:47 +0000 (0:00:00.037) 0:00:06.723 ******* 2020-11-20T13:51:47.730 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"ansible_facts": {"perform_users_role": false}, "changed": false} 2020-11-20T13:51:47.762 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:47 +0000 (0:00:00.037) 0:00:06.760 ******* 2020-11-20T13:51:47.767 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2020-11-20T13:51:47.768 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:47.801 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:47 +0000 (0:00:00.038) 0:00:06.799 ******* 2020-11-20T13:51:47.806 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:47.840 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:47 +0000 (0:00:00.039) 0:00:06.838 ******* 2020-11-20T13:51:47.845 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* 2020-11-20T13:51:47.847 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:47.877 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:47 +0000 (0:00:00.037) 0:00:06.876 ******* 2020-11-20T13:51:47.882 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:47.913 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:47 +0000 (0:00:00.036) 0:00:06.912 ******* 2020-11-20T13:51:47.918 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** 2020-11-20T13:51:47.919 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:47.951 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:47 +0000 (0:00:00.037) 0:00:06.949 ******* 2020-11-20T13:51:47.956 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:47.988 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:47 +0000 (0:00:00.037) 0:00:06.987 ******* 2020-11-20T13:51:47.994 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** 2020-11-20T13:51:47.995 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.027 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:48 +0000 (0:00:00.038) 0:00:07.025 ******* 2020-11-20T13:51:48.032 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.064 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:48 +0000 (0:00:00.037) 0:00:07.063 ******* 2020-11-20T13:51:48.069 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2020-11-20T13:51:48.071 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.103 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:48 +0000 (0:00:00.038) 0:00:07.102 ******* 2020-11-20T13:51:48.109 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.144 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:48 +0000 (0:00:00.040) 0:00:07.142 ******* 2020-11-20T13:51:48.151 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-11-20T13:51:48.152 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.163 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.164 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.174 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.176 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.186 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.188 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.198 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.199 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 2020-11-20T13:51:48.201 INFO:teuthology.task.ansible.out:8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.210 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.210 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.211 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.212 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEV 2020-11-20T13:51:48.214 INFO:teuthology.task.ansible.out:Gcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.223 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.224 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef 2020-11-20T13:51:48.226 INFO:teuthology.task.ansible.out:8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.235 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.237 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAA 2020-11-20T13:51:48.238 INFO:teuthology.task.ansible.out:B3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.248 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.249 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.250 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B9 2020-11-20T13:51:48.251 INFO:teuthology.task.ansible.out:35dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.260 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.262 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 2020-11-20T13:51:48.263 INFO:teuthology.task.ansible.out:02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.272 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.274 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin 2020-11-20T13:51:48.276 INFO:teuthology.task.ansible.out:@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7d 2020-11-20T13:51:48.278 INFO:teuthology.task.ansible.out:f5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.285 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.287 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.297 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.299 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.309 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.311 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da26 2020-11-20T13:51:48.313 INFO:teuthology.task.ansible.out:9942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b 2020-11-20T13:51:48.314 INFO:teuthology.task.ansible.out:2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.322 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.324 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.335 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.336 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.337 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.337 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", " 2020-11-20T13:51:48.338 INFO:teuthology.task.ansible.out:ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.347 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.348 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.348 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.349 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.349 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7 2020-11-20T13:51:48.351 INFO:teuthology.task.ansible.out:a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.361 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.362 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.373 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.374 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.375 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.385 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.387 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.387 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.398 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.398 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.399 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.399 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.401 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.410 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.412 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.412 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.423 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.425 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.425 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.435 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.437 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.437 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.448 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.449 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.460 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.462 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.462 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lm 2020-11-20T13:51:48.464 INFO:teuthology.task.ansible.out:g 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.473 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.475 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.475 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.486 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.487 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.487 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.498 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.500 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.500 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.511 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.512 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.513 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.523 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.525 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.525 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.536 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.537 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.538 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.548 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.550 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.560 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.562 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.573 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.574 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.585 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.586 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.587 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4 2020-11-20T13:51:48.588 INFO:teuthology.task.ansible.out:286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.598 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.599 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD 2020-11-20T13:51:48.601 INFO:teuthology.task.ansible.out:0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.609 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.611 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.622 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.623 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.624 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.635 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.635 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.637 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.647 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.649 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.649 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.660 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.660 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.661 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.661 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.663 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.673 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.675 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.685 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.687 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.687 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.698 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.699 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.700 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.710 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.712 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.712 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.713 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUgh 2020-11-20T13:51:48.714 INFO:teuthology.task.ansible.out:tas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.723 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.725 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.735 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.737 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.747 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.749 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.759 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.761 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.763 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.773 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.774 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.785 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.786 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.787 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.797 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.799 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.809 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.811 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.822 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.823 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.824 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.834 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.836 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.846 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.847 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.848 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.848 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var" 2020-11-20T13:51:48.850 INFO:teuthology.task.ansible.out:: "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.876 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.877 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.878 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.878 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.879 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:48.924 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:48 +0000 (0:00:00.780) 0:00:07.923 ******* 2020-11-20T13:51:48.953 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:48 +0000 (0:00:00.028) 0:00:07.951 ******* 2020-11-20T13:51:48.958 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2020-11-20T13:51:48.990 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:48 +0000 (0:00:00.037) 0:00:07.989 ******* 2020-11-20T13:51:49.027 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:49 +0000 (0:00:00.036) 0:00:08.025 ******* 2020-11-20T13:51:49.032 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2020-11-20T13:51:49.033 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.066 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:49 +0000 (0:00:00.039) 0:00:08.065 ******* 2020-11-20T13:51:49.072 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.107 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:49 +0000 (0:00:00.040) 0:00:08.106 ******* 2020-11-20T13:51:49.112 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2020-11-20T13:51:49.114 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.145 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:49 +0000 (0:00:00.037) 0:00:08.143 ******* 2020-11-20T13:51:49.150 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.182 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:49 +0000 (0:00:00.036) 0:00:08.180 ******* 2020-11-20T13:51:49.186 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2020-11-20T13:51:49.188 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.218 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:49 +0000 (0:00:00.036) 0:00:08.217 ******* 2020-11-20T13:51:49.223 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.256 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:49 +0000 (0:00:00.037) 0:00:08.254 ******* 2020-11-20T13:51:49.262 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-11-20T13:51:49.263 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.295 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:49 +0000 (0:00:00.039) 0:00:08.293 ******* 2020-11-20T13:51:49.301 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.334 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:49 +0000 (0:00:00.038) 0:00:08.332 ******* 2020-11-20T13:51:49.339 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** 2020-11-20T13:51:49.340 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.371 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:49 +0000 (0:00:00.036) 0:00:08.369 ******* 2020-11-20T13:51:49.376 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.410 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:49 +0000 (0:00:00.039) 0:00:08.409 ******* 2020-11-20T13:51:49.416 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-11-20T13:51:49.418 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.418 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.419 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.430 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.432 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.441 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.443 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.453 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.455 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.465 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.467 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.477 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.479 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.480 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.490 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.492 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.502 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.504 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.515 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.516 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.518 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.528 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.529 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.529 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.531 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.541 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.543 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.543 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.544 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.555 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.556 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.557 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.567 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.569 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.579 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.581 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.592 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.592 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.594 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.594 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.596 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.606 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.608 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.608 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.609 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.619 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.621 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.624 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.658 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:49 +0000 (0:00:00.247) 0:00:08.656 ******* 2020-11-20T13:51:49.664 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.664 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.666 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.666 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.667 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.667 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.667 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:49.708 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:49 +0000 (0:00:00.050) 0:00:08.706 ******* 2020-11-20T13:51:50.337 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2020-11-20T13:51:50.339 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "checksum": "eefb312a9cf82af5b9a46fb88fa640babe5fe20f", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/keys-repo-sha1", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 40, "state": "file", "uid": 0} 2020-11-20T13:51:50.370 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:50 +0000 (0:00:00.662) 0:00:09.369 ******* 2020-11-20T13:51:50.376 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"changed": false, "checksum": "eefb312a9cf82af5b9a46fb88fa640babe5fe20f", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/keys-repo-sha1", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 40, "state": "file", "uid": 0} 2020-11-20T13:51:50.409 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:50 +0000 (0:00:00.039) 0:00:09.408 ******* 2020-11-20T13:51:50.415 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2020-11-20T13:51:50.416 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => { "msg": "Host smithi116.front.sepia.ceph.com is running CentOS 8.1 (Core)" } 2020-11-20T13:51:50.450 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:50 +0000 (0:00:00.040) 0:00:09.448 ******* 2020-11-20T13:51:50.456 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => { "msg": "Host smithi040.front.sepia.ceph.com is running CentOS 8.1 (Core)" } 2020-11-20T13:51:50.490 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:50 +0000 (0:00:00.040) 0:00:09.488 ******* 2020-11-20T13:51:50.496 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2020-11-20T13:51:50.497 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml) => {"ansible_facts": {"container_mirror_cert_path": "/etc/containers/certs.d/{{ container_mirror }}", "nrpe_selinux_packages": ["python3-libsemanage", "python3-policycoreutils"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml"} 2020-11-20T13:51:50.530 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:50 +0000 (0:00:00.040) 0:00:09.529 ******* 2020-11-20T13:51:50.536 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml) => {"ansible_facts": {"container_mirror_cert_path": "/etc/containers/certs.d/{{ container_mirror }}", "nrpe_selinux_packages": ["python3-libsemanage", "python3-policycoreutils"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml"} 2020-11-20T13:51:50.569 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:50 +0000 (0:00:00.039) 0:00:09.568 ******* 2020-11-20T13:51:50.575 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2020-11-20T13:51:50.577 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:50.610 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:50 +0000 (0:00:00.039) 0:00:09.608 ******* 2020-11-20T13:51:50.615 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:50.649 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:50 +0000 (0:00:00.039) 0:00:09.648 ******* 2020-11-20T13:51:51.140 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2020-11-20T13:51:51.142 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "cmd": "timedatectl | grep -E \"Time ?zone\" | sed -e \"s/.*: \\(.*\\) (.*/\\1/\"", "delta": "0:00:00.338289", "end": "2020-11-20 13:51:51.116183", "rc": 0, "start": "2020-11-20 13:51:50.777894", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2020-11-20T13:51:51.174 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:51 +0000 (0:00:00.524) 0:00:10.172 ******* 2020-11-20T13:51:51.180 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"changed": false, "cmd": "timedatectl | grep -E \"Time ?zone\" | sed -e \"s/.*: \\(.*\\) (.*/\\1/\"", "delta": "0:00:00.330070", "end": "2020-11-20 13:51:51.157579", "rc": 0, "start": "2020-11-20 13:51:50.827509", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2020-11-20T13:51:51.213 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:51 +0000 (0:00:00.039) 0:00:10.212 ******* 2020-11-20T13:51:51.218 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2020-11-20T13:51:51.220 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:51.252 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:51 +0000 (0:00:00.038) 0:00:10.250 ******* 2020-11-20T13:51:51.257 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:51.288 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:51 +0000 (0:00:00.036) 0:00:10.287 ******* 2020-11-20T13:51:51.293 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2020-11-20T13:51:51.294 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:51.324 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:51 +0000 (0:00:00.036) 0:00:10.323 ******* 2020-11-20T13:51:51.329 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:51.363 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:51 +0000 (0:00:00.038) 0:00:10.362 ******* 2020-11-20T13:51:51.368 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2020-11-20T13:51:51.370 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:51.400 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:51 +0000 (0:00:00.037) 0:00:10.399 ******* 2020-11-20T13:51:51.405 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:51.437 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:51 +0000 (0:00:00.036) 0:00:10.435 ******* 2020-11-20T13:51:51.441 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2020-11-20T13:51:51.443 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:51.474 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:51 +0000 (0:00:00.036) 0:00:10.472 ******* 2020-11-20T13:51:51.478 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:51.511 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:51 +0000 (0:00:00.037) 0:00:10.510 ******* 2020-11-20T13:51:51.516 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2020-11-20T13:51:51.518 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:51.549 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:51 +0000 (0:00:00.038) 0:00:10.548 ******* 2020-11-20T13:51:51.553 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:51.586 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:51 +0000 (0:00:00.036) 0:00:10.584 ******* 2020-11-20T13:51:51.591 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2020-11-20T13:51:51.592 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:51.624 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:51 +0000 (0:00:00.038) 0:00:10.623 ******* 2020-11-20T13:51:51.629 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:51.662 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:51 +0000 (0:00:00.037) 0:00:10.660 ******* 2020-11-20T13:51:51.668 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2020-11-20T13:51:51.670 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi116.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T13:51:51.702 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:51 +0000 (0:00:00.039) 0:00:10.700 ******* 2020-11-20T13:51:51.707 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T13:51:51.708 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T13:51:51.740 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:51 +0000 (0:00:00.038) 0:00:10.738 ******* 2020-11-20T13:51:51.745 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2020-11-20T13:51:51.746 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:51.780 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:51 +0000 (0:00:00.040) 0:00:10.778 ******* 2020-11-20T13:51:51.785 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:51.819 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:51 +0000 (0:00:00.038) 0:00:10.817 ******* 2020-11-20T13:51:51.824 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2020-11-20T13:51:51.826 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:51.858 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:51 +0000 (0:00:00.039) 0:00:10.857 ******* 2020-11-20T13:51:51.863 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:51.897 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:51 +0000 (0:00:00.038) 0:00:10.895 ******* 2020-11-20T13:51:51.902 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2020-11-20T13:51:51.903 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T13:51:51.935 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:51 +0000 (0:00:00.038) 0:00:10.933 ******* 2020-11-20T13:51:51.940 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T13:51:51.973 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:51 +0000 (0:00:00.038) 0:00:10.971 ******* 2020-11-20T13:51:51.980 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2020-11-20T13:51:51.981 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:52.012 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:52 +0000 (0:00:00.039) 0:00:11.011 ******* 2020-11-20T13:51:52.017 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:52.051 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:52 +0000 (0:00:00.039) 0:00:11.050 ******* 2020-11-20T13:51:52.056 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2020-11-20T13:51:52.058 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T13:51:52.091 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:52 +0000 (0:00:00.039) 0:00:11.089 ******* 2020-11-20T13:51:52.103 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T13:51:52.135 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:52 +0000 (0:00:00.044) 0:00:11.134 ******* 2020-11-20T13:51:52.141 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2020-11-20T13:51:52.143 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:52.177 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:52 +0000 (0:00:00.041) 0:00:11.175 ******* 2020-11-20T13:51:52.182 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:52.216 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:52 +0000 (0:00:00.038) 0:00:11.214 ******* 2020-11-20T13:51:52.221 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2020-11-20T13:51:52.223 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:52.254 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:52 +0000 (0:00:00.038) 0:00:11.252 ******* 2020-11-20T13:51:52.259 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:52.292 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:52 +0000 (0:00:00.037) 0:00:11.290 ******* 2020-11-20T13:51:52.297 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2020-11-20T13:51:52.299 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:52.329 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:52 +0000 (0:00:00.037) 0:00:11.328 ******* 2020-11-20T13:51:52.334 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:52.367 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:52 +0000 (0:00:00.038) 0:00:11.366 ******* 2020-11-20T13:51:52.373 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2020-11-20T13:51:52.375 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:52.408 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:52 +0000 (0:00:00.040) 0:00:11.406 ******* 2020-11-20T13:51:52.413 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:52.446 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:52 +0000 (0:00:00.038) 0:00:11.445 ******* 2020-11-20T13:51:52.452 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2020-11-20T13:51:52.453 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T13:51:52.485 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:52 +0000 (0:00:00.038) 0:00:11.483 ******* 2020-11-20T13:51:52.490 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T13:51:52.524 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:52 +0000 (0:00:00.038) 0:00:11.522 ******* 2020-11-20T13:51:52.529 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-11-20T13:51:52.531 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:52.563 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:52 +0000 (0:00:00.039) 0:00:11.562 ******* 2020-11-20T13:51:52.568 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:52.602 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:52 +0000 (0:00:00.038) 0:00:11.600 ******* 2020-11-20T13:51:52.609 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2020-11-20T13:51:52.610 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:52.641 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:52 +0000 (0:00:00.039) 0:00:11.640 ******* 2020-11-20T13:51:52.646 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:52.679 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:52 +0000 (0:00:00.037) 0:00:11.678 ******* 2020-11-20T13:51:52.684 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2020-11-20T13:51:52.685 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:52.717 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:52 +0000 (0:00:00.038) 0:00:11.716 ******* 2020-11-20T13:51:52.722 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:52.755 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:52 +0000 (0:00:00.038) 0:00:11.754 ******* 2020-11-20T13:51:52.761 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2020-11-20T13:51:52.762 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:52.795 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:52 +0000 (0:00:00.039) 0:00:11.793 ******* 2020-11-20T13:51:52.800 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:52.833 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:52 +0000 (0:00:00.037) 0:00:11.831 ******* 2020-11-20T13:51:52.837 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2020-11-20T13:51:52.839 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:52.872 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:52 +0000 (0:00:00.039) 0:00:11.871 ******* 2020-11-20T13:51:52.877 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:52.911 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:52 +0000 (0:00:00.038) 0:00:11.910 ******* 2020-11-20T13:51:52.917 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2020-11-20T13:51:52.919 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:52.952 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:52 +0000 (0:00:00.040) 0:00:11.950 ******* 2020-11-20T13:51:52.958 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:52.992 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:52 +0000 (0:00:00.039) 0:00:11.990 ******* 2020-11-20T13:51:52.997 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2020-11-20T13:51:52.999 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T13:51:53.030 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:53 +0000 (0:00:00.038) 0:00:12.029 ******* 2020-11-20T13:51:53.035 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T13:51:53.067 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:53 +0000 (0:00:00.037) 0:00:12.066 ******* 2020-11-20T13:51:53.072 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2020-11-20T13:51:53.074 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:53.106 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:53 +0000 (0:00:00.038) 0:00:12.104 ******* 2020-11-20T13:51:53.111 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:53.144 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:53 +0000 (0:00:00.038) 0:00:12.142 ******* 2020-11-20T13:51:53.149 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2020-11-20T13:51:53.150 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:53.182 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:53 +0000 (0:00:00.038) 0:00:12.181 ******* 2020-11-20T13:51:53.187 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:53.221 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:53 +0000 (0:00:00.038) 0:00:12.219 ******* 2020-11-20T13:51:53.382 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2020-11-20T13:51:53.384 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-20T13:51:53.416 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:53 +0000 (0:00:00.195) 0:00:12.415 ******* 2020-11-20T13:51:53.443 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-20T13:51:53.476 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:53 +0000 (0:00:00.059) 0:00:12.474 ******* 2020-11-20T13:51:53.876 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2020-11-20T13:51:53.878 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "86c0d58bde7ad4cfaafe592d2fefd92c920c42ca", "dest": "/etc/yum.repos.d/epel.repo", "gid": 0, "group": "root", "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 236, "state": "file", "uid": 0} 2020-11-20T13:51:53.961 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "86c0d58bde7ad4cfaafe592d2fefd92c920c42ca", "dest": "/etc/yum.repos.d/epel.repo", "gid": 0, "group": "root", "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 236, "state": "file", "uid": 0} 2020-11-20T13:51:54.303 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "a1bad5509ed35b895c2061841236b1cfe10619ae", "dest": "/etc/yum.repos.d/epel-testing.repo", "gid": 0, "group": "root", "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 262, "state": "file", "uid": 0} 2020-11-20T13:51:54.339 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:54 +0000 (0:00:00.862) 0:00:13.337 ******* 2020-11-20T13:51:54.407 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "a1bad5509ed35b895c2061841236b1cfe10619ae", "dest": "/etc/yum.repos.d/epel-testing.repo", "gid": 0, "group": "root", "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 262, "state": "file", "uid": 0} 2020-11-20T13:51:54.456 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:54 +0000 (0:00:00.116) 0:00:13.454 ******* 2020-11-20T13:51:54.805 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2020-11-20T13:51:54.807 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "3c9ba7bacd1e30bed014cefd1fb4eab0f0f8fa10", "dest": "/etc/yum.repos.d/epel-mirrorlist", "gid": 0, "group": "root", "item": "epel-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 484, "state": "file", "uid": 0} 2020-11-20T13:51:54.930 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "3c9ba7bacd1e30bed014cefd1fb4eab0f0f8fa10", "dest": "/etc/yum.repos.d/epel-mirrorlist", "gid": 0, "group": "root", "item": "epel-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 484, "state": "file", "uid": 0} 2020-11-20T13:51:55.231 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "d542c3e2c30a0d3eb7d454775342c302c0ded15b", "dest": "/etc/yum.repos.d/epel-testing-mirrorlist", "gid": 0, "group": "root", "item": "epel-testing-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 540, "state": "file", "uid": 0} 2020-11-20T13:51:55.266 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:55 +0000 (0:00:00.810) 0:00:14.264 ******* 2020-11-20T13:51:55.292 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-11-20T13:51:55.292 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:55.324 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:55 +0000 (0:00:00.058) 0:00:14.323 ******* 2020-11-20T13:51:55.350 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-11-20T13:51:55.351 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:55.382 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:55 +0000 (0:00:00.057) 0:00:14.380 ******* 2020-11-20T13:51:55.397 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2020-11-20T13:51:55.399 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "d542c3e2c30a0d3eb7d454775342c302c0ded15b", "dest": "/etc/yum.repos.d/epel-testing-mirrorlist", "gid": 0, "group": "root", "item": "epel-testing-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 540, "state": "file", "uid": 0} 2020-11-20T13:51:55.432 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:55 +0000 (0:00:00.049) 0:00:14.430 ******* 2020-11-20T13:51:55.437 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2020-11-20T13:51:55.439 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:55.471 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:55 +0000 (0:00:00.039) 0:00:14.469 ******* 2020-11-20T13:51:55.476 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-11-20T13:51:55.477 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:55.509 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:55 +0000 (0:00:00.038) 0:00:14.508 ******* 2020-11-20T13:51:55.515 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2020-11-20T13:51:55.517 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:55.548 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:55 +0000 (0:00:00.038) 0:00:14.546 ******* 2020-11-20T13:51:55.553 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-11-20T13:51:55.555 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:55.587 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:55 +0000 (0:00:00.039) 0:00:14.585 ******* 2020-11-20T13:51:55.593 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-11-20T13:51:55.594 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:55.625 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:55 +0000 (0:00:00.038) 0:00:14.624 ******* 2020-11-20T13:51:55.631 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2020-11-20T13:51:55.633 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:55.663 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:55 +0000 (0:00:00.037) 0:00:14.662 ******* 2020-11-20T13:51:55.668 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2020-11-20T13:51:55.670 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:55.701 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:55 +0000 (0:00:00.037) 0:00:14.699 ******* 2020-11-20T13:51:55.706 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2020-11-20T13:51:55.707 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:55.739 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:55 +0000 (0:00:00.037) 0:00:14.737 ******* 2020-11-20T13:51:55.744 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2020-11-20T13:51:55.745 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:55.780 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:55 +0000 (0:00:00.041) 0:00:14.779 ******* 2020-11-20T13:51:55.786 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-11-20T13:51:55.788 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:55.818 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:55 +0000 (0:00:00.037) 0:00:14.816 ******* 2020-11-20T13:51:55.829 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2020-11-20T13:51:55.831 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:55.861 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:55 +0000 (0:00:00.042) 0:00:14.859 ******* 2020-11-20T13:51:55.866 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2020-11-20T13:51:55.867 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:55.898 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:55 +0000 (0:00:00.037) 0:00:14.897 ******* 2020-11-20T13:51:55.903 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2020-11-20T13:51:55.905 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:55.937 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:55 +0000 (0:00:00.038) 0:00:14.935 ******* 2020-11-20T13:51:55.942 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2020-11-20T13:51:55.944 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:55.978 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:55 +0000 (0:00:00.041) 0:00:14.976 ******* 2020-11-20T13:51:56.004 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2020-11-20T13:51:56.006 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:56.037 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:56 +0000 (0:00:00.058) 0:00:15.035 ******* 2020-11-20T13:51:56.063 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2020-11-20T13:51:56.064 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:56.094 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:56 +0000 (0:00:00.057) 0:00:15.093 ******* 2020-11-20T13:51:56.525 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2020-11-20T13:51:56.527 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.224972", "end": "2020-11-20 13:51:56.500392", "rc": 0, "start": "2020-11-20 13:51:56.275420", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} changed: [smithi116.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.393992", "end": "2020-11-20 13:51:56.500347", "rc": 0, "start": "2020-11-20 13:51:56.106355", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T13:51:56.557 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:56 +0000 (0:00:00.462) 0:00:15.555 ******* 2020-11-20T13:51:56.586 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:56 +0000 (0:00:00.029) 0:00:15.585 ******* 2020-11-20T13:51:57.720 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2020-11-20T13:51:57.722 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: krb5-workstation"]} 2020-11-20T13:51:57.753 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:57 +0000 (0:00:01.167) 0:00:16.752 ******* 2020-11-20T13:51:57.779 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2020-11-20T13:51:57.781 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Install Kerberos Packages (RedHat)] ***************************** ok: [smithi040.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: krb5-workstation"]} 2020-11-20T13:51:57.813 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:57 +0000 (0:00:00.059) 0:00:16.812 ******* 2020-11-20T13:51:57.845 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:57 +0000 (0:00:00.032) 0:00:16.844 ******* 2020-11-20T13:51:57.851 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2020-11-20T13:51:57.852 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:57.884 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:57 +0000 (0:00:00.038) 0:00:16.883 ******* 2020-11-20T13:51:57.890 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2020-11-20T13:51:57.891 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:57.923 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:57 +0000 (0:00:00.039) 0:00:16.922 ******* 2020-11-20T13:51:57.929 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:57.961 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:57 +0000 (0:00:00.037) 0:00:16.959 ******* 2020-11-20T13:51:57.966 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2020-11-20T13:51:57.968 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:58.000 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:57 +0000 (0:00:00.038) 0:00:16.998 ******* 2020-11-20T13:51:58.005 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:58.038 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:58 +0000 (0:00:00.038) 0:00:17.037 ******* 2020-11-20T13:51:58.044 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2020-11-20T13:51:58.045 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:58.077 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:58 +0000 (0:00:00.038) 0:00:17.076 ******* 2020-11-20T13:51:58.083 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:58.118 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:58 +0000 (0:00:00.041) 0:00:17.117 ******* 2020-11-20T13:51:58.509 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2020-11-20T13:51:58.511 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "secontext": "system_u:object_r:krb5_conf_t:s0", "size": 131, "state": "file", "uid": 0} 2020-11-20T13:51:58.542 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:58 +0000 (0:00:00.423) 0:00:17.540 ******* 2020-11-20T13:51:58.579 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "secontext": "system_u:object_r:krb5_conf_t:s0", "size": 131, "state": "file", "uid": 0} 2020-11-20T13:51:58.612 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:58 +0000 (0:00:00.070) 0:00:17.611 ******* 2020-11-20T13:52:01.802 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2020-11-20T13:52:01.804 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: smartmontools", "Installed: smartmontools-1:7.1-1.el8.x86_64", "Removed: smartmontools-1:6.6-3.el8.x86_64"]} 2020-11-20T13:52:01.836 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:01 +0000 (0:00:03.223) 0:00:20.834 ******* 2020-11-20T13:52:01.955 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: smartmontools", "Installed: smartmontools-1:7.1-1.el8.x86_64", "Removed: smartmontools-1:6.6-3.el8.x86_64"]} 2020-11-20T13:52:01.990 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:01 +0000 (0:00:00.153) 0:00:20.988 ******* 2020-11-20T13:52:02.431 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-11-20T13:52:02.433 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "secontext": "system_u:object_r:bin_t:s0", "size": 2716736, "state": "file", "uid": 0} 2020-11-20T13:52:02.505 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "secontext": "system_u:object_r:bin_t:s0", "size": 2716736, "state": "file", "uid": 0} 2020-11-20T13:52:02.930 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "secontext": "system_u:object_r:bin_t:s0", "size": 1952038, "state": "file", "uid": 0} 2020-11-20T13:52:03.014 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "secontext": "system_u:object_r:bin_t:s0", "size": 1952038, "state": "file", "uid": 0} 2020-11-20T13:52:03.388 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "secontext": "system_u:object_r:bin_t:s0", "size": 580432, "state": "file", "uid": 0} 2020-11-20T13:52:03.421 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:03 +0000 (0:00:01.431) 0:00:22.420 ******* 2020-11-20T13:52:03.447 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "secontext": "system_u:object_r:bin_t:s0", "size": 580432, "state": "file", "uid": 0} 2020-11-20T13:52:03.481 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:03 +0000 (0:00:00.059) 0:00:22.479 ******* 2020-11-20T13:52:03.609 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2020-11-20T13:52:03.611 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-11-20T13:52:03.644 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:03 +0000 (0:00:00.162) 0:00:22.642 ******* 2020-11-20T13:52:03.680 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-11-20T13:52:03.712 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:03 +0000 (0:00:00.068) 0:00:22.711 ******* 2020-11-20T13:52:04.092 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-11-20T13:52:04.093 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "secontext": "system_u:object_r:bin_t:s0", "size": 12725, "state": "file", "uid": 0} 2020-11-20T13:52:04.189 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "secontext": "system_u:object_r:bin_t:s0", "size": 12725, "state": "file", "uid": 0} 2020-11-20T13:52:04.511 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 8602, "state": "file", "uid": 0} 2020-11-20T13:52:04.637 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 8602, "state": "file", "uid": 0} 2020-11-20T13:52:04.917 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 3605, "state": "file", "uid": 0} 2020-11-20T13:52:04.952 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:04 +0000 (0:00:01.239) 0:00:23.951 ******* 2020-11-20T13:52:04.978 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2020-11-20T13:52:04.980 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:05.010 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:05 +0000 (0:00:00.057) 0:00:24.008 ******* 2020-11-20T13:52:05.035 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2020-11-20T13:52:05.037 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib64/nagios/plugins", "nrpe_group": "nrpe", "nrpe_packages": ["nagios-common", "nrpe", "nagios-plugins", "nagios-plugins-load"], "nrpe_service_name": "nrpe", "nrpe_user": "nrpe"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/yum_systems.yml"], "changed": false} 2020-11-20T13:52:05.067 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:05 +0000 (0:00:00.056) 0:00:24.065 ******* 2020-11-20T13:52:05.072 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-11-20T13:52:05.073 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 3605, "state": "file", "uid": 0} 2020-11-20T13:52:05.107 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:05 +0000 (0:00:00.039) 0:00:24.105 ******* 2020-11-20T13:52:05.132 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2020-11-20T13:52:05.134 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:05.164 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:05 +0000 (0:00:00.057) 0:00:24.163 ******* 2020-11-20T13:52:05.190 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2020-11-20T13:52:05.192 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib64/nagios/plugins", "nrpe_group": "nrpe", "nrpe_packages": ["nagios-common", "nrpe", "nagios-plugins", "nagios-plugins-load"], "nrpe_service_name": "nrpe", "nrpe_user": "nrpe"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/yum_systems.yml"], "changed": false} 2020-11-20T13:52:05.223 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:05 +0000 (0:00:00.058) 0:00:24.221 ******* 2020-11-20T13:52:05.249 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2020-11-20T13:52:05.250 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2020-11-20T13:52:05.280 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:05 +0000 (0:00:00.057) 0:00:24.279 ******* 2020-11-20T13:52:05.317 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2020-11-20T13:52:05.319 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-20T13:52:05.328 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-20T13:52:05.338 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-20T13:52:05.349 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-20T13:52:05.382 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:05 +0000 (0:00:00.101) 0:00:24.380 ******* 2020-11-20T13:52:06.311 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2020-11-20T13:52:06.313 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: nagios-common", "Installed: nrpe", "Installed: nagios-plugins", "Installed: nagios-plugins-load"]} 2020-11-20T13:52:06.343 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:06 +0000 (0:00:00.961) 0:00:25.342 ******* 2020-11-20T13:52:06.369 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2020-11-20T13:52:06.371 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:06.402 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:06 +0000 (0:00:00.058) 0:00:25.400 ******* 2020-11-20T13:52:06.427 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2020-11-20T13:52:06.429 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2020-11-20T13:52:06.460 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:06 +0000 (0:00:00.058) 0:00:25.458 ******* 2020-11-20T13:52:06.496 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2020-11-20T13:52:06.498 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-20T13:52:06.507 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-20T13:52:06.518 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-20T13:52:06.519 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-20T13:52:06.551 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:06 +0000 (0:00:00.091) 0:00:25.550 ******* 2020-11-20T13:52:06.838 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2020-11-20T13:52:06.840 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "checksum": "b423b985ccbb69b65a711d372645a5f704b30e8b", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "secontext": "system_u:object_r:etc_t:s0", "size": 178, "state": "file", "uid": 0} 2020-11-20T13:52:06.871 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:06 +0000 (0:00:00.319) 0:00:25.869 ******* 2020-11-20T13:52:07.309 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2020-11-20T13:52:07.311 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib64/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib64/nagios/plugins/check_mem.sh", "secontext": "system_u:object_r:nagios_unconfined_plugin_exec_t:s0", "size": 1964, "state": "file", "uid": 0} 2020-11-20T13:52:07.343 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:07 +0000 (0:00:00.472) 0:00:26.341 ******* 2020-11-20T13:52:07.369 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2020-11-20T13:52:07.370 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "DAEMON_OPTS", "skip_reason": "Conditional result was False"} 2020-11-20T13:52:07.380 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "NRPE_OPTS", "skip_reason": "Conditional result was False"} 2020-11-20T13:52:07.415 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:07 +0000 (0:00:00.071) 0:00:26.413 ******* 2020-11-20T13:52:07.554 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2020-11-20T13:52:07.556 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: nagios-common", "Installed: nrpe", "Installed: nagios-plugins", "Installed: nagios-plugins-load"]} 2020-11-20T13:52:07.588 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:07 +0000 (0:00:00.172) 0:00:26.586 ******* 2020-11-20T13:52:07.613 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:07.615 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* ok: [smithi116.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-20T13:52:07.647 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:07 +0000 (0:00:00.059) 0:00:26.645 ******* 2020-11-20T13:52:07.679 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:07 +0000 (0:00:00.031) 0:00:26.677 ******* 2020-11-20T13:52:07.911 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2020-11-20T13:52:07.913 INFO:teuthology.task.ansible.out:fatal: [smithi116.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} ...ignoring 2020-11-20T13:52:07.945 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:07 +0000 (0:00:00.266) 0:00:26.943 ******* 2020-11-20T13:52:07.971 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2020-11-20T13:52:07.972 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:08.005 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:07 +0000 (0:00:00.059) 0:00:27.003 ******* 2020-11-20T13:52:08.115 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2020-11-20T13:52:08.116 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"changed": false, "checksum": "b423b985ccbb69b65a711d372645a5f704b30e8b", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "secontext": "system_u:object_r:etc_t:s0", "size": 178, "state": "file", "uid": 0} 2020-11-20T13:52:08.149 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:08 +0000 (0:00:00.144) 0:00:27.147 ******* 2020-11-20T13:52:08.446 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2020-11-20T13:52:08.448 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "checksum": "aa9150d4582f919d7a8504c01c8d86f05e3eeb5d", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "secontext": "system_u:object_r:nrpe_etc_t:s0", "size": 1184, "state": "file", "uid": 0} 2020-11-20T13:52:08.482 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:08 +0000 (0:00:00.333) 0:00:27.480 ******* 2020-11-20T13:52:08.601 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2020-11-20T13:52:08.603 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib64/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib64/nagios/plugins/check_mem.sh", "secontext": "system_u:object_r:nagios_unconfined_plugin_exec_t:s0", "size": 1964, "state": "file", "uid": 0} 2020-11-20T13:52:08.635 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:08 +0000 (0:00:00.153) 0:00:27.633 ******* 2020-11-20T13:52:08.661 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2020-11-20T13:52:08.663 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "DAEMON_OPTS", "skip_reason": "Conditional result was False"} 2020-11-20T13:52:08.672 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "NRPE_OPTS", "skip_reason": "Conditional result was False"} 2020-11-20T13:52:08.705 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:08 +0000 (0:00:00.069) 0:00:27.703 ******* 2020-11-20T13:52:08.917 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2020-11-20T13:52:08.918 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-20T13:52:08.949 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:08 +0000 (0:00:00.244) 0:00:27.948 ******* 2020-11-20T13:52:09.173 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2020-11-20T13:52:09.175 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nrpe", "state": "started", "status": {"ActiveEnterTimestamp": "Fri 2020-11-20 13:50:49 UTC", "ActiveEnterTimestampMonotonic": "15676289", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "systemd-tmpfiles-setup.service sysinit.target local-fs.target -.mount basic.target tmp.mount nss-lookup.target var-run.mount time-sync.target system.slice systemd-journald.socket network.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Fri 2020-11-20 13:50:49 UTC", "AssertTimestampMonotonic": "15675460", "Before": "getty@tty1.service multi-user.target plymouth-quit.service xdm.service shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Fri 2020-11-20 13:50:49 UTC", "ConditionTimestampMonotonic": "15675459", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target nrpe.socket", "ControlGroup": "/system.slice/nrpe.service", "ControlPID": "0", "D 2020-11-20T13:52:09.177 INFO:teuthology.task.ansible.out:efaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "2487", "ExecMainStartTimestamp": "Fri 2020-11-20 13:50:49 UTC", "ExecMainStartTimestampMonotonic": "15676259", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nrpe.service", "GID": "986", "Group": "nrpe", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nrpe.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Fri 2020-11-20 13:50:49 UTC", "InactiveExitTimestampMonotonic": "15676289", "InvocationID": "52f96496de6a443fb252c2181ed68363", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitD 2020-11-20T13:52:09.178 INFO:teuthology.task.ansible.out:ATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127426", "LimitNPROCSoft": "127426", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127426", "LimitSIGPENDINGSoft": "127426", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "2487", "MemoryAccounting": "yes", "MemoryCurrent": "1171456", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nrpe.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nrpe/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice sysinit.target -.mount", "RequiresMountsFor": "/var/tmp /run/nrpe", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectory": "nrpe", "RuntimeDirectoryMode 2020-11-20T13:52:09.180 INFO:teuthology.task.ansible.out:": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Fri 2020-11-20 13:50:49 UTC", "StateChangeTimestampMonotonic": "15676289", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "990", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "nrpe", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Fri 2020-11-20 13:50:49 UTC", "WatchdogTimestampMonotonic": "15676288", "WatchdogUSec": "0"}} 2020-11-20T13:52:09.208 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:09 +0000 (0:00:00.259) 0:00:28.207 ******* 2020-11-20T13:52:09.214 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2020-11-20T13:52:09.215 INFO:teuthology.task.ansible.out:fatal: [smithi040.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} ...ignoring 2020-11-20T13:52:09.248 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:09 +0000 (0:00:00.039) 0:00:28.247 ******* 2020-11-20T13:52:09.274 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:09.310 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:09 +0000 (0:00:00.062) 0:00:28.309 ******* 2020-11-20T13:52:09.388 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2020-11-20T13:52:09.390 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => {"changed": true, "cmd": ["getenforce"], "delta": "0:00:00.001717", "end": "2020-11-20 13:52:09.372269", "rc": 0, "start": "2020-11-20 13:52:09.370552", "stderr": "", "stderr_lines": [], "stdout": "Permissive", "stdout_lines": ["Permissive"]} 2020-11-20T13:52:09.428 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:09 +0000 (0:00:00.117) 0:00:28.427 ******* 2020-11-20T13:52:09.778 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* ok: [smithi040.front.sepia.ceph.com] => {"changed": false, "checksum": "aa9150d4582f919d7a8504c01c8d86f05e3eeb5d", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "secontext": "system_u:object_r:nrpe_etc_t:s0", "size": 1184, "state": "file", "uid": 0} 2020-11-20T13:52:09.812 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:09 +0000 (0:00:00.383) 0:00:28.810 ******* 2020-11-20T13:52:10.128 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2020-11-20T13:52:10.129 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nrpe", "state": "started", "status": {"ActiveEnterTimestamp": "Fri 2020-11-20 13:50:34 UTC", "ActiveEnterTimestampMonotonic": "19520260", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "time-sync.target local-fs.target var-run.mount sysinit.target systemd-journald.socket network.target systemd-tmpfiles-setup.service system.slice basic.target nss-lookup.target -.mount tmp.mount", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Fri 2020-11-20 13:50:34 UTC", "AssertTimestampMonotonic": "19519462", "Before": "getty@tty1.service shutdown.target plymouth-quit.service xdm.service multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Fri 2020-11-20 13:50:34 UTC", "ConditionTimestampMonotonic": "19519462", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target nrpe.socket", "ControlGroup": "/system.slice/nrpe.service", "ControlPID": "0", "D 2020-11-20T13:52:10.130 INFO:teuthology.task.ansible.out:efaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "2494", "ExecMainStartTimestamp": "Fri 2020-11-20 13:50:34 UTC", "ExecMainStartTimestampMonotonic": "19520229", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nrpe.service", "GID": "986", "Group": "nrpe", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nrpe.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Fri 2020-11-20 13:50:34 UTC", "InactiveExitTimestampMonotonic": "19520260", "InvocationID": "b351fe628c6e4ccb87f46d9c0b14c0db", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitD 2020-11-20T13:52:10.132 INFO:teuthology.task.ansible.out:ATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127450", "LimitNPROCSoft": "127450", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127450", "LimitSIGPENDINGSoft": "127450", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "2494", "MemoryAccounting": "yes", "MemoryCurrent": "1232896", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nrpe.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nrpe/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target -.mount system.slice", "RequiresMountsFor": "/run/nrpe /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectory": "nrpe", "RuntimeDirectoryMode 2020-11-20T13:52:10.133 INFO:teuthology.task.ansible.out:": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Fri 2020-11-20 13:50:34 UTC", "StateChangeTimestampMonotonic": "19520260", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "990", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "nrpe", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Fri 2020-11-20 13:50:34 UTC", "WatchdogTimestampMonotonic": "19520260", "WatchdogUSec": "0"}} 2020-11-20T13:52:10.163 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:10 +0000 (0:00:00.351) 0:00:29.162 ******* 2020-11-20T13:52:10.355 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2020-11-20T13:52:10.357 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => {"changed": true, "cmd": ["getenforce"], "delta": "0:00:00.001784", "end": "2020-11-20 13:52:10.338069", "rc": 0, "start": "2020-11-20 13:52:10.336285", "stderr": "", "stderr_lines": [], "stdout": "Permissive", "stdout_lines": ["Permissive"]} 2020-11-20T13:52:10.388 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:10 +0000 (0:00:00.224) 0:00:29.386 ******* 2020-11-20T13:52:10.393 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2020-11-20T13:52:10.395 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: python3-libsemanage", "Installed: python3-policycoreutils"]} 2020-11-20T13:52:10.429 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:10 +0000 (0:00:00.040) 0:00:29.427 ******* 2020-11-20T13:52:11.353 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: python3-libsemanage", "Installed: python3-policycoreutils"]} 2020-11-20T13:52:11.386 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:11 +0000 (0:00:00.957) 0:00:30.385 ******* 2020-11-20T13:52:11.391 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2020-11-20T13:52:11.393 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: selinux-policy-targeted"]} 2020-11-20T13:52:11.426 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:11 +0000 (0:00:00.039) 0:00:30.424 ******* 2020-11-20T13:52:12.376 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: selinux-policy-targeted"]} 2020-11-20T13:52:12.411 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:12 +0000 (0:00:00.985) 0:00:31.410 ******* 2020-11-20T13:52:12.843 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2020-11-20T13:52:12.844 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "name": "nagios_run_sudo", "persistent": true, "state": true} 2020-11-20T13:52:12.877 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:12 +0000 (0:00:00.465) 0:00:31.875 ******* 2020-11-20T13:52:13.484 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"changed": false, "name": "nagios_run_sudo", "persistent": true, "state": true} 2020-11-20T13:52:13.519 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:13 +0000 (0:00:00.642) 0:00:32.517 ******* 2020-11-20T13:52:29.363 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2020-11-20T13:52:29.365 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-r", "nrpe"], "delta": "0:00:16.290592", "end": "2020-11-20 13:52:29.344142", "failed_when_result": false, "rc": 0, "start": "2020-11-20 13:52:13.053550", "stderr": "", "stderr_lines": [], "stdout": "libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority).", "stdout_lines": ["libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority)."]} 2020-11-20T13:52:29.397 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:29 +0000 (0:00:15.877) 0:00:48.395 ******* 2020-11-20T13:52:29.838 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2020-11-20T13:52:29.840 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "checksum": "99d9b2ea41fb2d6613898129718064833d2de428", "dest": "/tmp/nrpe.te", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/tmp/nrpe.te", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 934, "state": "file", "uid": 0} 2020-11-20T13:52:29.871 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:29 +0000 (0:00:00.474) 0:00:48.870 ******* 2020-11-20T13:52:30.083 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2020-11-20T13:52:30.085 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => {"changed": true, "cmd": ["checkmodule", "-M", "-m", "-o", "/tmp/nrpe.mod", "/tmp/nrpe.te"], "delta": "0:00:00.019774", "end": "2020-11-20 13:52:30.060525", "rc": 0, "start": "2020-11-20 13:52:30.040751", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T13:52:30.117 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:30 +0000 (0:00:00.245) 0:00:49.115 ******* 2020-11-20T13:52:30.318 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2020-11-20T13:52:30.320 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule_package", "-o", "/tmp/nrpe.pp", "-m", "/tmp/nrpe.mod"], "delta": "0:00:00.012782", "end": "2020-11-20 13:52:30.299772", "rc": 0, "start": "2020-11-20 13:52:30.286990", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T13:52:30.351 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:30 +0000 (0:00:00.234) 0:00:49.350 ******* 2020-11-20T13:52:30.886 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2020-11-20T13:52:30.888 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-r", "nrpe"], "delta": "0:00:17.163317", "end": "2020-11-20 13:52:30.860960", "failed_when_result": false, "rc": 0, "start": "2020-11-20 13:52:13.697643", "stderr": "", "stderr_lines": [], "stdout": "libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority).", "stdout_lines": ["libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority)."]} 2020-11-20T13:52:30.921 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:30 +0000 (0:00:00.569) 0:00:49.920 ******* 2020-11-20T13:52:31.404 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2020-11-20T13:52:31.406 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"changed": false, "checksum": "99d9b2ea41fb2d6613898129718064833d2de428", "dest": "/tmp/nrpe.te", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/tmp/nrpe.te", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 934, "state": "file", "uid": 0} 2020-11-20T13:52:31.441 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:31 +0000 (0:00:00.519) 0:00:50.439 ******* 2020-11-20T13:52:31.653 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2020-11-20T13:52:31.655 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => {"changed": true, "cmd": ["checkmodule", "-M", "-m", "-o", "/tmp/nrpe.mod", "/tmp/nrpe.te"], "delta": "0:00:00.010591", "end": "2020-11-20 13:52:31.628794", "rc": 0, "start": "2020-11-20 13:52:31.618203", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T13:52:31.689 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:31 +0000 (0:00:00.247) 0:00:50.687 ******* 2020-11-20T13:52:31.901 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2020-11-20T13:52:31.903 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule_package", "-o", "/tmp/nrpe.pp", "-m", "/tmp/nrpe.mod"], "delta": "0:00:00.017226", "end": "2020-11-20 13:52:31.883703", "rc": 0, "start": "2020-11-20 13:52:31.866477", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T13:52:31.933 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:31 +0000 (0:00:00.244) 0:00:50.931 ******* 2020-11-20T13:52:46.783 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2020-11-20T13:52:46.785 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-i", "/tmp/nrpe.pp"], "delta": "0:00:16.234368", "end": "2020-11-20 13:52:46.758085", "rc": 0, "start": "2020-11-20 13:52:30.523717", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T13:52:46.816 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:46 +0000 (0:00:14.883) 0:01:05.815 ******* 2020-11-20T13:52:47.010 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2020-11-20T13:52:47.012 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "path": "/tmp/nrpe.*", "state": "absent"} 2020-11-20T13:52:47.042 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:47 +0000 (0:00:00.225) 0:01:06.041 ******* 2020-11-20T13:52:47.068 INFO:teuthology.task.ansible.out: TASK [common : Check for podman] *********************************************** 2020-11-20T13:52:47.070 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:47.100 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:47 +0000 (0:00:00.058) 0:01:06.099 ******* 2020-11-20T13:52:47.136 INFO:teuthology.task.ansible.out: TASK [common : set_fact] ******************************************************* 2020-11-20T13:52:47.138 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"ansible_facts": {"container_mirror_cert_path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000"}, "changed": false} 2020-11-20T13:52:47.171 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:47 +0000 (0:00:00.070) 0:01:06.169 ******* 2020-11-20T13:52:47.197 INFO:teuthology.task.ansible.out: TASK [common : Check for docker] *********************************************** 2020-11-20T13:52:47.198 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:47.229 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:47 +0000 (0:00:00.057) 0:01:06.227 ******* 2020-11-20T13:52:47.254 INFO:teuthology.task.ansible.out: TASK [common : set_fact] ******************************************************* 2020-11-20T13:52:47.256 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:47.287 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:47 +0000 (0:00:00.058) 0:01:06.286 ******* 2020-11-20T13:52:47.333 INFO:teuthology.task.ansible.out: TASK [common : Include encrypted variables] ************************************ 2020-11-20T13:52:47.335 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} ok: [smithi116.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T13:52:47.368 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:47 +0000 (0:00:00.081) 0:01:06.367 ******* 2020-11-20T13:52:47.581 INFO:teuthology.task.ansible.out: TASK [common : Create /etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000] *** 2020-11-20T13:52:47.583 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => {"changed": true, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-11-20T13:52:47.617 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:47 +0000 (0:00:00.248) 0:01:06.615 ******* 2020-11-20T13:52:48.099 INFO:teuthology.task.ansible.out: TASK [common : Copy docker-mirror.front.sepia.ceph.com:5000 self-signed cert] *** 2020-11-20T13:52:48.101 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => {"changed": true, "checksum": "d386a5d594d8bb6c408cf08541a4eb609577a03d", "dest": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "md5sum": "e2cf8b6a8ad65ba91ea9fc53d49f1bf4", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 2236, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1605880367.6436849-213571886954283/source", "state": "file", "uid": 0} 2020-11-20T13:52:48.132 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:48 +0000 (0:00:00.515) 0:01:07.131 ******* 2020-11-20T13:52:48.158 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2020-11-20T13:52:48.160 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"ansible_facts": {"ceph_dependency_packages_to_remove": ["boost-random", "boost-program-options", "leveldb", "xmlstarlet", "python-jinja2", "python-ceph", "python-flask", "python-requests", "boost-random", "python-urllib3", "python-babel", "hdparm", "python-markupsafe", "python-werkzeug", "python-itsdangerous"], "ceph_packages_to_remove": ["ceph", "ceph-base", "ceph-selinux", "ceph-common", "ceph-debuginfo", "ceph-release", "libcephfs1", "ceph-radosgw", "python-ceph", "python-rados", "python-rbd", "python-cephfs", "librbd1", "librados2", "mod_fastcgi"], "ntp_service_name": "ntpd", "packages_to_remove": ["device-mapper-multipath"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "sshd"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/dnf_systems.yml"], "changed": false} 2020-11-20T13:52:48.191 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:48 +0000 (0:00:00.058) 0:01:07.190 ******* 2020-11-20T13:52:48.227 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2020-11-20T13:52:48.229 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-11-20T13:52:48.260 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:48 +0000 (0:00:00.068) 0:01:07.258 ******* 2020-11-20T13:52:48.297 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2020-11-20T13:52:48.299 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml) => {"ansible_facts": {"common_yum_repos": {"lab-extras": {"baseurl": "http://{{ mirror_host }}/lab-extras/8/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "epel_packages": [], "nfs_service": "nfs-server", "ntp_service_name": "chronyd", "packages": ["dnf-utils", "sysstat", "libedit", "boost-thread", "xfsprogs", "gdisk", "parted", "libgcrypt", "fuse-libs", "openssl", "libuuid", "podman", "podman-docker", "attr", "ant", "lsof", "gettext", "bc", "xfsdump", "blktrace", "usbredir", "libev-devel", "valgrind", "nfs-utils", "ncurses-devel", "gcc", "git", "python3-nose", "python3-virtualenv", "genisoimage", "qemu-img", "qemu-kvm-core", "qemu-kvm-block-rbd", "libacl-devel", "dbench", "autoconf", "gdb", "iozone"], "yum_mirrorlists": ["CentOS-AppStream-mirrorlist", "CentOS-Base-mirrorlist", "CentOS-Extras-mirrorlist"], "yum_repos": {"CentOS-AppStream": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist", "name": "CentOS-$releasever - AppStream"}, "CentOS-Base": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Base-mirrorlist", "name": "CentOS-$releasever - Base"}, "CentOS-Extras": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Extras-mirrorlist", "name": "CentOS-$releasever - Extras"}}}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml"} 2020-11-20T13:52:48.330 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:48 +0000 (0:00:00.070) 0:01:07.329 ******* 2020-11-20T13:52:48.367 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2020-11-20T13:52:48.368 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-11-20T13:52:48.399 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:48 +0000 (0:00:00.068) 0:01:07.398 ******* 2020-11-20T13:52:48.581 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2020-11-20T13:52:48.583 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2020-11-20T13:52:48.617 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:48 +0000 (0:00:00.217) 0:01:07.615 ******* 2020-11-20T13:52:48.808 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2020-11-20T13:52:48.810 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "name": "ubuntu", "state": "present", "system": false} 2020-11-20T13:52:48.841 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:48 +0000 (0:00:00.224) 0:01:07.840 ******* 2020-11-20T13:52:48.867 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2020-11-20T13:52:48.869 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:48.901 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:48 +0000 (0:00:00.059) 0:01:07.899 ******* 2020-11-20T13:52:48.928 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2020-11-20T13:52:48.929 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:48.974 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:48 +0000 (0:00:00.072) 0:01:07.972 ******* 2020-11-20T13:52:49.179 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2020-11-20T13:52:49.181 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/bash", "state": "present", "uid": 10101} 2020-11-20T13:52:49.213 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:49 +0000 (0:00:00.239) 0:01:08.212 ******* 2020-11-20T13:52:49.239 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2020-11-20T13:52:49.242 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-i", "/tmp/nrpe.pp"], "delta": "0:00:17.098675", "end": "2020-11-20 13:52:49.213079", "rc": 0, "start": "2020-11-20 13:52:32.114404", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T13:52:49.275 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:49 +0000 (0:00:00.061) 0:01:08.273 ******* 2020-11-20T13:52:49.478 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2020-11-20T13:52:49.480 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"changed": false, "path": "/tmp/nrpe.*", "state": "absent"} 2020-11-20T13:52:49.511 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:49 +0000 (0:00:00.236) 0:01:08.510 ******* 2020-11-20T13:52:49.538 INFO:teuthology.task.ansible.out: TASK [common : Check for podman] *********************************************** skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:49.573 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:49 +0000 (0:00:00.060) 0:01:08.570 ******* 2020-11-20T13:52:49.608 INFO:teuthology.task.ansible.out: TASK [common : set_fact] ******************************************************* 2020-11-20T13:52:49.611 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"ansible_facts": {"container_mirror_cert_path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000"}, "changed": false} 2020-11-20T13:52:49.642 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:49 +0000 (0:00:00.070) 0:01:08.641 ******* 2020-11-20T13:52:49.670 INFO:teuthology.task.ansible.out: TASK [common : Check for docker] *********************************************** skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:49.702 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:49 +0000 (0:00:00.059) 0:01:08.700 ******* 2020-11-20T13:52:49.728 INFO:teuthology.task.ansible.out: TASK [common : set_fact] ******************************************************* 2020-11-20T13:52:49.730 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:49.761 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:49 +0000 (0:00:00.059) 0:01:08.759 ******* 2020-11-20T13:52:49.768 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2020-11-20T13:52:49.769 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 112, "state": "file", "uid": 0} 2020-11-20T13:52:49.803 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:49 +0000 (0:00:00.042) 0:01:08.802 ******* 2020-11-20T13:52:49.820 INFO:teuthology.task.ansible.out: TASK [common : Include encrypted variables] ************************************ 2020-11-20T13:52:49.821 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T13:52:49.822 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T13:52:49.855 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:49 +0000 (0:00:00.052) 0:01:08.854 ******* 2020-11-20T13:52:49.862 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2020-11-20T13:52:49.863 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi116"}, "changed": false} 2020-11-20T13:52:49.897 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:49 +0000 (0:00:00.040) 0:01:08.895 ******* 2020-11-20T13:52:50.092 INFO:teuthology.task.ansible.out: TASK [common : Create /etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000] *** 2020-11-20T13:52:50.094 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => {"changed": true, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-11-20T13:52:50.127 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:50 +0000 (0:00:00.230) 0:01:09.125 ******* 2020-11-20T13:52:50.644 INFO:teuthology.task.ansible.out: TASK [common : Copy docker-mirror.front.sepia.ceph.com:5000 self-signed cert] *** 2020-11-20T13:52:50.646 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => {"changed": true, "checksum": "d386a5d594d8bb6c408cf08541a4eb609577a03d", "dest": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "md5sum": "e2cf8b6a8ad65ba91ea9fc53d49f1bf4", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 2236, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1605880370.1552544-231349417006918/source", "state": "file", "uid": 0} 2020-11-20T13:52:50.677 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:50 +0000 (0:00:00.549) 0:01:09.675 ******* 2020-11-20T13:52:50.703 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2020-11-20T13:52:50.704 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"ansible_facts": {"ceph_dependency_packages_to_remove": ["boost-random", "boost-program-options", "leveldb", "xmlstarlet", "python-jinja2", "python-ceph", "python-flask", "python-requests", "boost-random", "python-urllib3", "python-babel", "hdparm", "python-markupsafe", "python-werkzeug", "python-itsdangerous"], "ceph_packages_to_remove": ["ceph", "ceph-base", "ceph-selinux", "ceph-common", "ceph-debuginfo", "ceph-release", "libcephfs1", "ceph-radosgw", "python-ceph", "python-rados", "python-rbd", "python-cephfs", "librbd1", "librados2", "mod_fastcgi"], "ntp_service_name": "ntpd", "packages_to_remove": ["device-mapper-multipath"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "sshd"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/dnf_systems.yml"], "changed": false} 2020-11-20T13:52:50.735 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:50 +0000 (0:00:00.058) 0:01:09.734 ******* 2020-11-20T13:52:50.772 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2020-11-20T13:52:50.773 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-11-20T13:52:50.805 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:50 +0000 (0:00:00.069) 0:01:09.803 ******* 2020-11-20T13:52:50.843 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2020-11-20T13:52:50.846 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml) => {"ansible_facts": {"common_yum_repos": {"lab-extras": {"baseurl": "http://{{ mirror_host }}/lab-extras/8/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "epel_packages": [], "nfs_service": "nfs-server", "ntp_service_name": "chronyd", "packages": ["dnf-utils", "sysstat", "libedit", "boost-thread", "xfsprogs", "gdisk", "parted", "libgcrypt", "fuse-libs", "openssl", "libuuid", "podman", "podman-docker", "attr", "ant", "lsof", "gettext", "bc", "xfsdump", "blktrace", "usbredir", "libev-devel", "valgrind", "nfs-utils", "ncurses-devel", "gcc", "git", "python3-nose", "python3-virtualenv", "genisoimage", "qemu-img", "qemu-kvm-core", "qemu-kvm-block-rbd", "libacl-devel", "dbench", "autoconf", "gdb", "iozone"], "yum_mirrorlists": ["CentOS-AppStream-mirrorlist", "CentOS-Base-mirrorlist", "CentOS-Extras-mirrorlist"], "yum_repos": {"CentOS-AppStream": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist", "name": "CentOS-$releasever - AppStream"}, "CentOS-Base": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Base-mirrorlist", "name": "CentOS-$releasever - Base"}, "CentOS-Extras": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Extras-mirrorlist", "name": "CentOS-$releasever - Extras"}}}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml"} 2020-11-20T13:52:50.878 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:50 +0000 (0:00:00.073) 0:01:09.876 ******* 2020-11-20T13:52:50.883 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2020-11-20T13:52:50.885 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "", "ansible_fqdn": "smithi116", "ansible_hostname": "smithi116", "ansible_nodename": "smithi116"}, "changed": false, "name": "smithi116"} 2020-11-20T13:52:50.918 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:50 +0000 (0:00:00.040) 0:01:09.917 ******* 2020-11-20T13:52:50.924 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2020-11-20T13:52:50.926 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-11-20T13:52:50.959 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:50 +0000 (0:00:00.040) 0:01:09.958 ******* 2020-11-20T13:52:51.162 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2020-11-20T13:52:51.164 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2020-11-20T13:52:51.194 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:51 +0000 (0:00:00.234) 0:01:10.193 ******* 2020-11-20T13:52:51.355 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2020-11-20T13:52:51.357 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "checksum": "a5e550b2aa74f6355e926d70d73a50ad9ab4b4be", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "secontext": "system_u:object_r:etc_t:s0", "size": 1043, "state": "file", "uid": 0} 2020-11-20T13:52:51.389 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:51 +0000 (0:00:00.194) 0:01:10.388 ******* 2020-11-20T13:52:51.395 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2020-11-20T13:52:51.396 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "name": "ubuntu", "state": "present", "system": false} 2020-11-20T13:52:51.431 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:51 +0000 (0:00:00.041) 0:01:10.429 ******* 2020-11-20T13:52:51.457 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2020-11-20T13:52:51.458 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:51.490 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:51 +0000 (0:00:00.058) 0:01:10.488 ******* 2020-11-20T13:52:51.515 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2020-11-20T13:52:51.517 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:51.551 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:51 +0000 (0:00:00.061) 0:01:10.550 ******* 2020-11-20T13:52:51.775 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2020-11-20T13:52:51.777 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/bash", "state": "present", "uid": 10101} 2020-11-20T13:52:51.824 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:51 +0000 (0:00:00.272) 0:01:10.822 ******* 2020-11-20T13:52:51.840 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2020-11-20T13:52:51.842 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "secontext": "system_u:object_r:etc_t:s0", "size": 570, "state": "file", "uid": 0} 2020-11-20T13:52:51.876 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:51 +0000 (0:00:00.051) 0:01:10.874 ******* 2020-11-20T13:52:52.357 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2020-11-20T13:52:52.359 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 112, "state": "file", "uid": 0} 2020-11-20T13:52:52.391 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:52 +0000 (0:00:00.515) 0:01:11.390 ******* 2020-11-20T13:52:52.418 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2020-11-20T13:52:52.419 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi040"}, "changed": false} 2020-11-20T13:52:52.451 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:52 +0000 (0:00:00.060) 0:01:11.450 ******* 2020-11-20T13:52:52.605 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2020-11-20T13:52:52.606 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"attempts": 1, "changed": false, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2020-11-20T13:52:52.639 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:52 +0000 (0:00:00.187) 0:01:11.638 ******* 2020-11-20T13:52:53.039 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2020-11-20T13:52:53.041 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "", "ansible_fqdn": "smithi040", "ansible_hostname": "smithi040", "ansible_nodename": "smithi040"}, "changed": false, "name": "smithi040"} 2020-11-20T13:52:53.074 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:53 +0000 (0:00:00.435) 0:01:12.073 ******* 2020-11-20T13:52:53.091 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2020-11-20T13:52:53.092 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "checksum": "68d0b8ee2f88a77895f257aa69f32a42e47f455b", "dest": "/etc/security/limits.d/remote.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.d/remote.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 116, "state": "file", "uid": 0} 2020-11-20T13:52:53.126 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:53 +0000 (0:00:00.052) 0:01:12.125 ******* 2020-11-20T13:52:53.319 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-11-20T13:52:53.321 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "dest": "/etc/fuse.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 0, "state": "file", "uid": 0} 2020-11-20T13:52:53.352 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:53 +0000 (0:00:00.225) 0:01:12.351 ******* 2020-11-20T13:52:53.544 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-11-20T13:52:53.546 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "gid": 36, "name": "kvm", "state": "present", "system": false} TASK [testnode : Upload distro major version specific sshd_config] ************* ok: [smithi040.front.sepia.ceph.com] => {"changed": false, "checksum": "a5e550b2aa74f6355e926d70d73a50ad9ab4b4be", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "secontext": "system_u:object_r:etc_t:s0", "size": 1043, "state": "file", "uid": 0} 2020-11-20T13:52:53.577 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:53 +0000 (0:00:00.224) 0:01:12.575 ******* 2020-11-20T13:52:53.606 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:53 +0000 (0:00:00.028) 0:01:12.604 ******* 2020-11-20T13:52:53.811 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-11-20T13:52:53.813 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1236, "groups": "kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-11-20T13:52:53.846 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:53 +0000 (0:00:00.240) 0:01:12.845 ******* 2020-11-20T13:52:54.038 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2020-11-20T13:52:54.040 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "secontext": "system_u:object_r:etc_t:s0", "size": 570, "state": "file", "uid": 0} 2020-11-20T13:52:54.077 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:54 +0000 (0:00:00.230) 0:01:13.075 ******* 2020-11-20T13:52:54.292 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-11-20T13:52:54.294 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "checksum": "be3223c8b0917bde02657a5f9ddd95277cc3f812", "dest": "/etc/sudoers", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers", "secontext": "system_u:object_r:etc_t:s0", "size": 1808, "state": "file", "uid": 0} 2020-11-20T13:52:54.327 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:54 +0000 (0:00:00.250) 0:01:13.325 ******* 2020-11-20T13:52:54.707 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2020-11-20T13:52:54.709 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"attempts": 1, "changed": false, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2020-11-20T13:52:54.741 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:54 +0000 (0:00:00.414) 0:01:13.740 ******* 2020-11-20T13:52:54.779 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-11-20T13:52:54.781 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "checksum": "12a2e17405b9e00ea2061e25929905d08e5fe19f", "dest": "/etc/security/limits.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 854, "state": "file", "uid": 0} 2020-11-20T13:52:54.819 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:54 +0000 (0:00:00.077) 0:01:13.817 ******* 2020-11-20T13:52:55.219 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2020-11-20T13:52:55.220 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"changed": false, "checksum": "68d0b8ee2f88a77895f257aa69f32a42e47f455b", "dest": "/etc/security/limits.d/remote.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.d/remote.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 116, "state": "file", "uid": 0} 2020-11-20T13:52:55.254 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:55 +0000 (0:00:00.434) 0:01:14.252 ******* 2020-11-20T13:52:55.459 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** ok: [smithi040.front.sepia.ceph.com] => {"changed": false, "dest": "/etc/fuse.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 0, "state": "file", "uid": 0} 2020-11-20T13:52:55.492 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:55 +0000 (0:00:00.237) 0:01:14.490 ******* 2020-11-20T13:52:55.694 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-11-20T13:52:55.696 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"changed": false, "gid": 36, "name": "kvm", "state": "present", "system": false} 2020-11-20T13:52:55.727 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:55 +0000 (0:00:00.235) 0:01:14.725 ******* 2020-11-20T13:52:55.950 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-11-20T13:52:55.951 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1236, "groups": "kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-11-20T13:52:55.987 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:55 +0000 (0:00:00.260) 0:01:14.986 ******* 2020-11-20T13:52:56.470 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-11-20T13:52:56.472 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"changed": false, "checksum": "be3223c8b0917bde02657a5f9ddd95277cc3f812", "dest": "/etc/sudoers", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers", "secontext": "system_u:object_r:etc_t:s0", "size": 1808, "state": "file", "uid": 0} 2020-11-20T13:52:56.503 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:56 +0000 (0:00:00.515) 0:01:15.502 ******* 2020-11-20T13:52:56.644 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-11-20T13:52:56.646 INFO:teuthology.task.ansible.out:[WARNING]: Consider using the yum, dnf or zypper module rather than running 'rpm'. If you need to use command because yum, dnf or zypper is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. changed: [smithi116.front.sepia.ceph.com] => {"changed": true, "cmd": ["rpm", "--rebuilddb"], "delta": "0:00:01.639983", "end": "2020-11-20 13:52:56.622513", "rc": 0, "start": "2020-11-20 13:52:54.982530", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T13:52:56.684 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:56 +0000 (0:00:00.180) 0:01:15.682 ******* 2020-11-20T13:52:56.982 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-11-20T13:52:56.984 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"changed": false, "checksum": "12a2e17405b9e00ea2061e25929905d08e5fe19f", "dest": "/etc/security/limits.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 854, "state": "file", "uid": 0} 2020-11-20T13:52:57.017 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:57 +0000 (0:00:00.333) 0:01:16.016 ******* 2020-11-20T13:52:57.161 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2020-11-20T13:52:57.163 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => (item=CentOS-AppStream-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "aa5ad73629677ef17172417e50d2cdadccb7a3f2", "dest": "/etc/yum.repos.d/CentOS-AppStream-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-AppStream-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-AppStream-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 397, "state": "file", "uid": 0} 2020-11-20T13:52:57.588 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => (item=CentOS-Base-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "4ed3c0272e6c4ec99169cd68f0b43d40bdd4b296", "dest": "/etc/yum.repos.d/CentOS-Base-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-Base-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Base-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 380, "state": "file", "uid": 0} 2020-11-20T13:52:58.014 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => (item=CentOS-Extras-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "a862f43d3b3c7b8b592aa7dce23a6d013ab20bc3", "dest": "/etc/yum.repos.d/CentOS-Extras-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-Extras-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Extras-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 382, "state": "file", "uid": 0} 2020-11-20T13:52:58.050 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:58 +0000 (0:00:01.032) 0:01:17.049 ******* 2020-11-20T13:52:58.517 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2020-11-20T13:52:58.519 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => (item={'key': 'lab-extras', 'value': {'name': 'lab-extras', 'baseurl': 'http://apt-mirror.front.sepia.ceph.com/lab-extras/8/', 'enabled': 1, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "a0e2ac0628a264e14b96b1c57627a3aa344a6c52", "dest": "/etc/yum.repos.d/lab-extras.repo", "gid": 0, "group": "root", "item": {"key": "lab-extras", "value": {"baseurl": "http://apt-mirror.front.sepia.ceph.com/lab-extras/8/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/lab-extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 203, "state": "file", "uid": 0} 2020-11-20T13:52:58.554 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:58 +0000 (0:00:00.503) 0:01:17.553 ******* 2020-11-20T13:52:58.892 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-11-20T13:52:58.894 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => {"changed": true, "cmd": ["rpm", "--rebuilddb"], "delta": "0:00:01.671667", "end": "2020-11-20 13:52:58.865888", "rc": 0, "start": "2020-11-20 13:52:57.194221", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T13:52:58.929 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:58 +0000 (0:00:00.374) 0:01:17.927 ******* 2020-11-20T13:52:59.020 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** ok: [smithi116.front.sepia.ceph.com] => (item={'key': 'CentOS-AppStream', 'value': {'name': 'CentOS-$releasever - AppStream', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "9b147c879e95cd23f3bbde027da102febcaad1a8", "dest": "/etc/yum.repos.d/CentOS-AppStream.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-AppStream", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist", "name": "CentOS-$releasever - AppStream"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-AppStream.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 289, "state": "file", "uid": 0} 2020-11-20T13:52:59.427 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2020-11-20T13:52:59.428 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => (item=CentOS-AppStream-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "aa5ad73629677ef17172417e50d2cdadccb7a3f2", "dest": "/etc/yum.repos.d/CentOS-AppStream-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-AppStream-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-AppStream-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 397, "state": "file", "uid": 0} 2020-11-20T13:52:59.449 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2020-11-20T13:52:59.450 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => (item={'key': 'CentOS-Base', 'value': {'name': 'CentOS-$releasever - Base', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-Base-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "77e4590ed0ecec056fa44f4e65af60e110822b85", "dest": "/etc/yum.repos.d/CentOS-Base.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-Base", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Base-mirrorlist", "name": "CentOS-$releasever - Base"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Base.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 274, "state": "file", "uid": 0} 2020-11-20T13:52:59.880 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => (item={'key': 'CentOS-Extras', 'value': {'name': 'CentOS-$releasever - Extras', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-Extras-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "2856826614c7f1cb9409228f6c41319a023ca754", "dest": "/etc/yum.repos.d/CentOS-Extras.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-Extras", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Extras-mirrorlist", "name": "CentOS-$releasever - Extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 280, "state": "file", "uid": 0} 2020-11-20T13:52:59.885 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2020-11-20T13:52:59.887 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => (item=CentOS-Base-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "4ed3c0272e6c4ec99169cd68f0b43d40bdd4b296", "dest": "/etc/yum.repos.d/CentOS-Base-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-Base-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Base-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 380, "state": "file", "uid": 0} 2020-11-20T13:52:59.935 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:59 +0000 (0:00:01.006) 0:01:18.933 ******* 2020-11-20T13:52:59.962 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2020-11-20T13:52:59.998 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:59 +0000 (0:00:00.062) 0:01:18.996 ******* 2020-11-20T13:53:00.024 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2020-11-20T13:53:00.026 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:00.058 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:00 +0000 (0:00:00.060) 0:01:19.057 ******* 2020-11-20T13:53:00.096 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2020-11-20T13:53:00.098 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-11-20T13:53:00.134 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:00 +0000 (0:00:00.075) 0:01:19.132 ******* 2020-11-20T13:53:00.162 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2020-11-20T13:53:00.164 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:00.211 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:00 +0000 (0:00:00.076) 0:01:19.209 ******* 2020-11-20T13:53:00.238 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-20T13:53:00.240 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"ansible_facts": {"skip_packaging": false}, "changed": false} 2020-11-20T13:53:00.270 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:00 +0000 (0:00:00.059) 0:01:19.269 ******* 2020-11-20T13:53:00.296 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2020-11-20T13:53:00.298 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:00.332 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:00 +0000 (0:00:00.061) 0:01:19.330 ******* 2020-11-20T13:53:00.340 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2020-11-20T13:53:00.342 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => (item=CentOS-Extras-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "a862f43d3b3c7b8b592aa7dce23a6d013ab20bc3", "dest": "/etc/yum.repos.d/CentOS-Extras-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-Extras-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Extras-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 382, "state": "file", "uid": 0} 2020-11-20T13:53:00.379 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:00 +0000 (0:00:00.046) 0:01:19.377 ******* 2020-11-20T13:53:00.385 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2020-11-20T13:53:00.387 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:00.422 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:00 +0000 (0:00:00.043) 0:01:19.420 ******* 2020-11-20T13:53:00.626 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2020-11-20T13:53:00.627 INFO:teuthology.task.ansible.out:fatal: [smithi116.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": ["rpm", "-q", "ceph-debuginfo"], "delta": "0:00:00.006705", "end": "2020-11-20 13:53:00.602807", "msg": "non-zero return code", "rc": 1, "start": "2020-11-20 13:53:00.596102", "stderr": "", "stderr_lines": [], "stdout": "package ceph-debuginfo is not installed", "stdout_lines": ["package ceph-debuginfo is not installed"]} ...ignoring 2020-11-20T13:53:00.663 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:00 +0000 (0:00:00.240) 0:01:19.661 ******* 2020-11-20T13:53:00.701 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2020-11-20T13:53:00.704 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:00.740 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:00 +0000 (0:00:00.076) 0:01:19.738 ******* 2020-11-20T13:53:00.882 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2020-11-20T13:53:00.883 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => (item={'key': 'lab-extras', 'value': {'name': 'lab-extras', 'baseurl': 'http://apt-mirror.front.sepia.ceph.com/lab-extras/8/', 'enabled': 1, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "a0e2ac0628a264e14b96b1c57627a3aa344a6c52", "dest": "/etc/yum.repos.d/lab-extras.repo", "gid": 0, "group": "root", "item": {"key": "lab-extras", "value": {"baseurl": "http://apt-mirror.front.sepia.ceph.com/lab-extras/8/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/lab-extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 203, "state": "file", "uid": 0} 2020-11-20T13:53:00.922 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:00 +0000 (0:00:00.182) 0:01:19.920 ******* 2020-11-20T13:53:01.407 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2020-11-20T13:53:01.409 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => (item={'key': 'CentOS-AppStream', 'value': {'name': 'CentOS-$releasever - AppStream', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "9b147c879e95cd23f3bbde027da102febcaad1a8", "dest": "/etc/yum.repos.d/CentOS-AppStream.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-AppStream", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist", "name": "CentOS-$releasever - AppStream"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-AppStream.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 289, "state": "file", "uid": 0} 2020-11-20T13:53:01.859 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => (item={'key': 'CentOS-Base', 'value': {'name': 'CentOS-$releasever - Base', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-Base-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "77e4590ed0ecec056fa44f4e65af60e110822b85", "dest": "/etc/yum.repos.d/CentOS-Base.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-Base", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Base-mirrorlist", "name": "CentOS-$releasever - Base"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Base.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 274, "state": "file", "uid": 0} 2020-11-20T13:53:02.308 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => (item={'key': 'CentOS-Extras', 'value': {'name': 'CentOS-$releasever - Extras', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-Extras-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "2856826614c7f1cb9409228f6c41319a023ca754", "dest": "/etc/yum.repos.d/CentOS-Extras.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-Extras", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Extras-mirrorlist", "name": "CentOS-$releasever - Extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 280, "state": "file", "uid": 0} 2020-11-20T13:53:02.344 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:02 +0000 (0:00:01.422) 0:01:21.343 ******* 2020-11-20T13:53:02.370 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2020-11-20T13:53:02.403 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:02 +0000 (0:00:00.058) 0:01:21.401 ******* 2020-11-20T13:53:02.433 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2020-11-20T13:53:02.435 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:02.467 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:02 +0000 (0:00:00.064) 0:01:21.465 ******* 2020-11-20T13:53:02.504 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2020-11-20T13:53:02.506 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-11-20T13:53:02.538 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:02 +0000 (0:00:00.071) 0:01:21.537 ******* 2020-11-20T13:53:02.564 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2020-11-20T13:53:02.566 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:02.597 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:02 +0000 (0:00:00.059) 0:01:21.596 ******* 2020-11-20T13:53:02.624 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-20T13:53:02.625 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"ansible_facts": {"skip_packaging": false}, "changed": false} 2020-11-20T13:53:02.657 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:02 +0000 (0:00:00.059) 0:01:21.656 ******* 2020-11-20T13:53:02.684 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2020-11-20T13:53:02.686 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:02.719 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:02 +0000 (0:00:00.061) 0:01:21.718 ******* 2020-11-20T13:53:02.756 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2020-11-20T13:53:02.758 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:02.791 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:02 +0000 (0:00:00.071) 0:01:21.790 ******* 2020-11-20T13:53:02.954 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-11-20T13:53:02.956 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Removed: qemu-kvm-block-rbd-15:2.12.0-88.module_el8.1.0+297+df420408.3.x86_64", "Removed: librados2-1:12.2.7-9.el8.x86_64", "Removed: librbd1-1:12.2.7-9.el8.x86_64"]} 2020-11-20T13:53:02.991 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:02 +0000 (0:00:00.199) 0:01:21.989 ******* 2020-11-20T13:53:03.018 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2020-11-20T13:53:03.020 INFO:teuthology.task.ansible.out:fatal: [smithi040.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": ["rpm", "-q", "ceph-debuginfo"], "delta": "0:00:00.007173", "end": "2020-11-20 13:53:02.990882", "msg": "non-zero return code", "rc": 1, "start": "2020-11-20 13:53:02.983709", "stderr": "", "stderr_lines": [], "stdout": "package ceph-debuginfo is not installed", "stdout_lines": ["package ceph-debuginfo is not installed"]} ...ignoring 2020-11-20T13:53:03.054 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:03 +0000 (0:00:00.062) 0:01:22.052 ******* 2020-11-20T13:53:03.080 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2020-11-20T13:53:03.082 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:03.114 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:03 +0000 (0:00:00.059) 0:01:22.112 ******* 2020-11-20T13:53:04.910 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-11-20T13:53:04.912 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Removed: boost-program-options-1.66.0-6.el8.x86_64", "Removed: boost-random-1.66.0-6.el8.x86_64"]} 2020-11-20T13:53:04.945 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:04 +0000 (0:00:01.831) 0:01:23.943 ******* 2020-11-20T13:53:05.586 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-11-20T13:53:05.588 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Removed: qemu-kvm-block-rbd-15:2.12.0-88.module_el8.1.0+297+df420408.3.x86_64", "Removed: librados2-1:12.2.7-9.el8.x86_64", "Removed: librbd1-1:12.2.7-9.el8.x86_64"]} 2020-11-20T13:53:05.624 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:05 +0000 (0:00:00.678) 0:01:24.622 ******* 2020-11-20T13:53:07.604 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-11-20T13:53:07.606 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Removed: boost-program-options-1.66.0-6.el8.x86_64", "Removed: boost-random-1.66.0-6.el8.x86_64"]} 2020-11-20T13:53:07.638 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:07 +0000 (0:00:02.014) 0:01:26.636 ******* 2020-11-20T13:53:08.008 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-20T13:53:08.010 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: dnf-utils", "Installed: sysstat", "Installed: libedit", "Installed: boost-thread", "Installed: xfsprogs", "Installed: gdisk", "Installed: parted", "Installed: libgcrypt", "Installed: fuse-libs", "Installed: openssl", "Installed: libuuid", "Installed: podman", "Installed: podman-docker", "Installed: attr", "Installed: ant", "Installed: lsof", "Installed: gettext", "Installed: bc", "Installed: xfsdump", "Installed: blktrace", "Installed: usbredir", "Installed: libev-devel", "Installed: valgrind", "Installed: nfs-utils", "Installed: ncurses-devel", "Installed: gcc", "Installed: git", "Installed: python3-nose", "Installed: python3-virtualenv", "Installed: genisoimage", "Installed: qemu-img", "Installed: qemu-kvm-core", "Installed: qemu-kvm-block-rbd", "Installed: libacl-devel", "Installed: dbench", "Installed: autoconf", "Installed: gdb", "Installed: iozone", "Installed: boost-random-1.66.0-6.el8.x86_64", "Installed: qemu-kvm-block-rbd-15:2.12.0-88.module_el8.1.0+297+df420408.3.x86_64", "Installed: librados2-1:12.2.7-9.el8.x86_64", "Installed: librbd1-1:12.2.7-9.el8.x86_64", "Installed: boost-program-options-1.66.0-6.el8.x86_64"]} 2020-11-20T13:53:08.064 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:08 +0000 (0:00:00.425) 0:01:27.062 ******* 2020-11-20T13:53:08.091 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2020-11-20T13:53:08.093 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:08.125 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:08 +0000 (0:00:00.060) 0:01:27.123 ******* 2020-11-20T13:53:09.040 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-11-20T13:53:09.041 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": []} 2020-11-20T13:53:09.086 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:09 +0000 (0:00:00.961) 0:01:28.085 ******* 2020-11-20T13:53:09.115 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-20T13:53:09.117 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:09.157 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:09 +0000 (0:00:00.070) 0:01:28.155 ******* 2020-11-20T13:53:09.571 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2020-11-20T13:53:09.573 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "enabled": false, "name": "firewalld", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "basic.target system.slice dbus.service polkit.service dbus.socket sysinit.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target network-pre.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "BusName": "org.fedoraproject.FirewallD1", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConfigurationDirectoryMode": "0755", "Conflicts": "iptables.service ebtables.service ip6tables.service shutdown.target ipset.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "firewalld - dynamic firewall daemon", "DevicePolicy": "auto", "Documentation": "man:firewalld(1)", "DynamicUser": "no", "EnvironmentFiles": "/etc/sysconfig/firewalld (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTim 2020-11-20T13:53:09.575 INFO:teuthology.task.ansible.out:estampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/firewalld ; argv[]=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/firewalld.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "firewalld.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "mixed", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127426", "LimitNPROCSoft": "127426", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127426", 2020-11-20T13:53:09.576 INFO:teuthology.task.ansible.out:"LimitSIGPENDINGSoft": "127426", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "0", "MemoryAccounting": "yes", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "firewalld.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice dbus.socket sysinit.target", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "null", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "null", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestampMonotonic": "0", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "SyslogFacility": "3 2020-11-20T13:53:09.578 INFO:teuthology.task.ansible.out:", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "[not set]", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "dbus", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "UtmpMode": "init", "Wants": "network-pre.target", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2020-11-20T13:53:09.604 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:09 +0000 (0:00:00.447) 0:01:28.603 ******* 2020-11-20T13:53:09.889 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2020-11-20T13:53:09.891 INFO:teuthology.task.ansible.out:fatal: [smithi116.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Could not find the requested service iptables: host"} ...ignoring 2020-11-20T13:53:09.925 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:09 +0000 (0:00:00.320) 0:01:28.923 ******* 2020-11-20T13:53:10.502 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2020-11-20T13:53:10.504 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"} 2020-11-20T13:53:10.535 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:10 +0000 (0:00:00.609) 0:01:29.533 ******* 2020-11-20T13:53:10.561 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* 2020-11-20T13:53:10.563 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:10.594 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:10 +0000 (0:00:00.059) 0:01:29.593 ******* 2020-11-20T13:53:10.620 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** 2020-11-20T13:53:10.622 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:10.658 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:10 +0000 (0:00:00.063) 0:01:29.657 ******* 2020-11-20T13:53:10.684 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2020-11-20T13:53:10.686 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:10.717 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:10 +0000 (0:00:00.058) 0:01:29.716 ******* 2020-11-20T13:53:10.723 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-20T13:53:10.724 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: dnf-utils", "Installed: sysstat", "Installed: libedit", "Installed: boost-thread", "Installed: xfsprogs", "Installed: gdisk", "Installed: parted", "Installed: libgcrypt", "Installed: fuse-libs", "Installed: openssl", "Installed: libuuid", "Installed: podman", "Installed: podman-docker", "Installed: attr", "Installed: ant", "Installed: lsof", "Installed: gettext", "Installed: bc", "Installed: xfsdump", "Installed: blktrace", "Installed: usbredir", "Installed: libev-devel", "Installed: valgrind", "Installed: nfs-utils", "Installed: ncurses-devel", "Installed: gcc", "Installed: git", "Installed: python3-nose", "Installed: python3-virtualenv", "Installed: genisoimage", "Installed: qemu-img", "Installed: qemu-kvm-core", "Installed: qemu-kvm-block-rbd", "Installed: libacl-devel", "Installed: dbench", "Installed: autoconf", "Installed: gdb", "Installed: iozone", "Installed: boost-random-1.66.0-6.el8.x86_64", "Installed: qemu-kvm-block-rbd-15:2.12.0-88.module_el8.1.0+297+df420408.3.x86_64", "Installed: librados2-1:12.2.7-9.el8.x86_64", "Installed: librbd1-1:12.2.7-9.el8.x86_64", "Installed: boost-program-options-1.66.0-6.el8.x86_64"]} 2020-11-20T13:53:10.759 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:10 +0000 (0:00:00.041) 0:01:29.757 ******* 2020-11-20T13:53:10.764 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2020-11-20T13:53:10.766 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:10.810 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:10 +0000 (0:00:00.051) 0:01:29.808 ******* 2020-11-20T13:53:10.815 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2020-11-20T13:53:10.817 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:10.851 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:10 +0000 (0:00:00.041) 0:01:29.850 ******* 2020-11-20T13:53:10.859 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2020-11-20T13:53:10.860 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:10.894 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:10 +0000 (0:00:00.042) 0:01:29.892 ******* 2020-11-20T13:53:10.919 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2020-11-20T13:53:10.921 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:10.952 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:10 +0000 (0:00:00.058) 0:01:29.951 ******* 2020-11-20T13:53:10.979 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2020-11-20T13:53:11.012 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:11 +0000 (0:00:00.059) 0:01:30.010 ******* 2020-11-20T13:53:11.038 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2020-11-20T13:53:11.040 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:11.077 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:11 +0000 (0:00:00.065) 0:01:30.075 ******* 2020-11-20T13:53:11.102 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2020-11-20T13:53:11.104 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:11.134 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:11 +0000 (0:00:00.057) 0:01:30.133 ******* 2020-11-20T13:53:11.161 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2020-11-20T13:53:11.162 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-11-20T13:53:11.172 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2020-11-20T13:53:11.205 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:11 +0000 (0:00:00.070) 0:01:30.204 ******* 2020-11-20T13:53:11.231 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2020-11-20T13:53:11.233 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:11.264 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:11 +0000 (0:00:00.058) 0:01:30.262 ******* 2020-11-20T13:53:11.290 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2020-11-20T13:53:11.323 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:11 +0000 (0:00:00.059) 0:01:30.322 ******* 2020-11-20T13:53:11.349 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:11.383 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:11 +0000 (0:00:00.059) 0:01:30.381 ******* 2020-11-20T13:53:11.409 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2020-11-20T13:53:11.410 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:11.447 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:11 +0000 (0:00:00.064) 0:01:30.446 ******* 2020-11-20T13:53:11.473 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2020-11-20T13:53:11.475 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:11.505 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:11 +0000 (0:00:00.057) 0:01:30.503 ******* 2020-11-20T13:53:11.531 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-20T13:53:11.532 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:11.564 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:11 +0000 (0:00:00.058) 0:01:30.562 ******* 2020-11-20T13:53:11.590 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2020-11-20T13:53:11.591 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:11.623 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:11 +0000 (0:00:00.059) 0:01:30.622 ******* 2020-11-20T13:53:11.649 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-20T13:53:11.651 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:11.684 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:11 +0000 (0:00:00.060) 0:01:30.682 ******* 2020-11-20T13:53:11.712 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2020-11-20T13:53:11.714 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:11.745 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:11 +0000 (0:00:00.061) 0:01:30.743 ******* 2020-11-20T13:53:11.762 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2020-11-20T13:53:11.764 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:11.795 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:11 +0000 (0:00:00.049) 0:01:30.793 ******* 2020-11-20T13:53:11.820 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2020-11-20T13:53:11.822 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:11.852 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:11 +0000 (0:00:00.057) 0:01:30.850 ******* 2020-11-20T13:53:11.857 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-11-20T13:53:11.859 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": []} 2020-11-20T13:53:11.903 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:11 +0000 (0:00:00.050) 0:01:30.901 ******* 2020-11-20T13:53:11.909 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2020-11-20T13:53:11.910 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:11.944 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:11 +0000 (0:00:00.040) 0:01:30.942 ******* 2020-11-20T13:53:11.948 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-20T13:53:11.950 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:11.983 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:11 +0000 (0:00:00.039) 0:01:30.982 ******* 2020-11-20T13:53:11.989 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2020-11-20T13:53:11.990 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:12.022 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:12 +0000 (0:00:00.039) 0:01:31.021 ******* 2020-11-20T13:53:12.048 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2020-11-20T13:53:12.050 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:12.081 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:12 +0000 (0:00:00.058) 0:01:31.079 ******* 2020-11-20T13:53:12.106 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2020-11-20T13:53:12.108 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:12.139 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:12 +0000 (0:00:00.057) 0:01:31.137 ******* 2020-11-20T13:53:12.164 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2020-11-20T13:53:12.166 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:12.198 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:12 +0000 (0:00:00.058) 0:01:31.196 ******* 2020-11-20T13:53:12.223 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-11-20T13:53:12.225 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:12.254 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:12 +0000 (0:00:00.056) 0:01:31.253 ******* 2020-11-20T13:53:12.280 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-11-20T13:53:12.281 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:12.314 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:12 +0000 (0:00:00.059) 0:01:31.313 ******* 2020-11-20T13:53:12.320 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2020-11-20T13:53:12.322 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"changed": false, "enabled": false, "name": "firewalld", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "system.slice dbus.service polkit.service sysinit.target basic.target dbus.socket", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target network-pre.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "BusName": "org.fedoraproject.FirewallD1", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConfigurationDirectoryMode": "0755", "Conflicts": "ipset.service ebtables.service iptables.service shutdown.target ip6tables.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "firewalld - dynamic firewall daemon", "DevicePolicy": "auto", "Documentation": "man:firewalld(1)", "DynamicUser": "no", "EnvironmentFiles": "/etc/sysconfig/firewalld (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTim 2020-11-20T13:53:12.323 INFO:teuthology.task.ansible.out:estampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/firewalld ; argv[]=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/firewalld.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "firewalld.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "mixed", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127450", "LimitNPROCSoft": "127450", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127450", 2020-11-20T13:53:12.325 INFO:teuthology.task.ansible.out:"LimitSIGPENDINGSoft": "127450", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "0", "MemoryAccounting": "yes", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "firewalld.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice sysinit.target dbus.socket", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "null", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "null", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestampMonotonic": "0", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "SyslogFacility": "3 2020-11-20T13:53:12.326 INFO:teuthology.task.ansible.out:", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "[not set]", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "dbus", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "UtmpMode": "init", "Wants": "network-pre.target", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2020-11-20T13:53:12.358 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:12 +0000 (0:00:00.043) 0:01:31.356 ******* 2020-11-20T13:53:12.364 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-11-20T13:53:12.365 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:12.400 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:12 +0000 (0:00:00.042) 0:01:31.399 ******* 2020-11-20T13:53:12.426 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-11-20T13:53:12.428 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:12.461 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:12 +0000 (0:00:00.060) 0:01:31.459 ******* 2020-11-20T13:53:12.487 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-11-20T13:53:12.489 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:12.522 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:12 +0000 (0:00:00.061) 0:01:31.520 ******* 2020-11-20T13:53:12.548 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-11-20T13:53:12.550 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:12.582 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:12 +0000 (0:00:00.060) 0:01:31.580 ******* 2020-11-20T13:53:12.609 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-11-20T13:53:12.611 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:12.643 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:12 +0000 (0:00:00.060) 0:01:31.641 ******* 2020-11-20T13:53:12.658 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2020-11-20T13:53:12.660 INFO:teuthology.task.ansible.out:fatal: [smithi040.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Could not find the requested service iptables: host"} ...ignoring 2020-11-20T13:53:12.692 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:12 +0000 (0:00:00.049) 0:01:31.691 ******* 2020-11-20T13:53:12.699 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-11-20T13:53:12.700 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:12.735 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:12 +0000 (0:00:00.042) 0:01:31.733 ******* 2020-11-20T13:53:12.767 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-20T13:53:12.769 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:12.807 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:12 +0000 (0:00:00.072) 0:01:31.805 ******* 2020-11-20T13:53:12.835 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-11-20T13:53:12.836 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:12.869 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:12 +0000 (0:00:00.061) 0:01:31.867 ******* 2020-11-20T13:53:12.896 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:12.928 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:12 +0000 (0:00:00.059) 0:01:31.927 ******* 2020-11-20T13:53:12.954 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-20T13:53:12.955 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:12.988 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:12 +0000 (0:00:00.060) 0:01:31.987 ******* 2020-11-20T13:53:12.994 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2020-11-20T13:53:12.995 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"} 2020-11-20T13:53:13.029 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:13 +0000 (0:00:00.040) 0:01:32.028 ******* 2020-11-20T13:53:13.035 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-11-20T13:53:13.036 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:13.084 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:13 +0000 (0:00:00.055) 0:01:32.083 ******* 2020-11-20T13:53:13.090 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* 2020-11-20T13:53:13.092 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:13.125 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:13 +0000 (0:00:00.040) 0:01:32.123 ******* 2020-11-20T13:53:13.151 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:13.186 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:13 +0000 (0:00:00.060) 0:01:32.184 ******* 2020-11-20T13:53:13.212 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2020-11-20T13:53:13.214 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:13.251 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:13 +0000 (0:00:00.064) 0:01:32.249 ******* 2020-11-20T13:53:13.277 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2020-11-20T13:53:13.279 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:13.311 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:13 +0000 (0:00:00.060) 0:01:32.310 ******* 2020-11-20T13:53:13.318 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-20T13:53:13.320 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/bin/lspci", "gid": 0, "group": "root", "mode": "0777", "owner": "root", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 11, "src": "/sbin/lspci", "state": "link", "uid": 0} 2020-11-20T13:53:13.354 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:13 +0000 (0:00:00.042) 0:01:32.353 ******* 2020-11-20T13:53:13.359 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2020-11-20T13:53:13.361 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:13.400 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:13 +0000 (0:00:00.045) 0:01:32.398 ******* 2020-11-20T13:53:13.428 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2020-11-20T13:53:13.430 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:13.461 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:13 +0000 (0:00:00.061) 0:01:32.460 ******* 2020-11-20T13:53:13.480 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2020-11-20T13:53:13.513 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:13 +0000 (0:00:00.051) 0:01:32.511 ******* 2020-11-20T13:53:13.539 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:13.573 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:13 +0000 (0:00:00.060) 0:01:32.572 ******* 2020-11-20T13:53:13.580 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-20T13:53:13.581 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/ltp/testcases/bin", "secontext": "unconfined_u:object_r:lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-11-20T13:53:13.618 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:13 +0000 (0:00:00.044) 0:01:32.616 ******* 2020-11-20T13:53:13.623 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2020-11-20T13:53:13.625 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:13.662 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:13 +0000 (0:00:00.044) 0:01:32.660 ******* 2020-11-20T13:53:13.689 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2020-11-20T13:53:13.691 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-11-20T13:53:13.701 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2020-11-20T13:53:13.737 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:13 +0000 (0:00:00.075) 0:01:32.736 ******* 2020-11-20T13:53:13.764 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2020-11-20T13:53:13.766 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:13.801 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:13 +0000 (0:00:00.063) 0:01:32.800 ******* 2020-11-20T13:53:13.818 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-20T13:53:13.820 INFO:teuthology.task.ansible.out:[WARNING]: Cannot set fs attributes on a non-existent symlink target. follow should be set to False to avoid this. ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/lib/ltp/testcases/bin/fsstress", "src": "/usr/bin/fsstress"} 2020-11-20T13:53:13.853 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:13 +0000 (0:00:00.051) 0:01:32.852 ******* 2020-11-20T13:53:13.858 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2020-11-20T13:53:13.892 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:13 +0000 (0:00:00.038) 0:01:32.890 ******* 2020-11-20T13:53:13.898 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-11-20T13:53:13.899 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:13.931 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:13 +0000 (0:00:00.039) 0:01:32.930 ******* 2020-11-20T13:53:13.937 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2020-11-20T13:53:13.939 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:13.971 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:13 +0000 (0:00:00.039) 0:01:32.970 ******* 2020-11-20T13:53:13.977 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-11-20T13:53:13.979 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:14.013 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:14 +0000 (0:00:00.041) 0:01:33.011 ******* 2020-11-20T13:53:14.019 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2020-11-20T13:53:14.020 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:14.053 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:14 +0000 (0:00:00.039) 0:01:33.051 ******* 2020-11-20T13:53:14.059 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-11-20T13:53:14.061 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:14.099 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:14 +0000 (0:00:00.045) 0:01:33.097 ******* 2020-11-20T13:53:14.104 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2020-11-20T13:53:14.106 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:14.138 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:14 +0000 (0:00:00.039) 0:01:33.137 ******* 2020-11-20T13:53:14.144 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-20T13:53:14.145 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:14.178 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:14 +0000 (0:00:00.039) 0:01:33.177 ******* 2020-11-20T13:53:14.184 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-20T13:53:14.186 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:14.218 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:14 +0000 (0:00:00.039) 0:01:33.216 ******* 2020-11-20T13:53:14.225 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:14.258 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:14 +0000 (0:00:00.039) 0:01:33.256 ******* 2020-11-20T13:53:14.264 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2020-11-20T13:53:14.266 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:14.299 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:14 +0000 (0:00:00.041) 0:01:33.297 ******* 2020-11-20T13:53:14.305 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-20T13:53:14.307 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:14.340 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:14 +0000 (0:00:00.040) 0:01:33.338 ******* 2020-11-20T13:53:14.345 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-20T13:53:14.347 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:14.379 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:14 +0000 (0:00:00.039) 0:01:33.378 ******* 2020-11-20T13:53:14.385 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-20T13:53:14.387 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:14.419 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:14 +0000 (0:00:00.039) 0:01:33.418 ******* 2020-11-20T13:53:14.425 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2020-11-20T13:53:14.426 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:14.458 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:14 +0000 (0:00:00.039) 0:01:33.457 ******* 2020-11-20T13:53:14.472 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-20T13:53:14.474 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:14.504 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:14 +0000 (0:00:00.046) 0:01:33.503 ******* 2020-11-20T13:53:14.511 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2020-11-20T13:53:14.513 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:14.546 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:14 +0000 (0:00:00.041) 0:01:33.544 ******* 2020-11-20T13:53:14.552 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-20T13:53:14.553 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:14.585 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:14 +0000 (0:00:00.039) 0:01:33.584 ******* 2020-11-20T13:53:14.590 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2020-11-20T13:53:14.592 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:14.625 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:14 +0000 (0:00:00.039) 0:01:33.623 ******* 2020-11-20T13:53:14.631 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2020-11-20T13:53:14.632 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:14.665 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:14 +0000 (0:00:00.040) 0:01:33.663 ******* 2020-11-20T13:53:14.671 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2020-11-20T13:53:14.672 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:14.704 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:14 +0000 (0:00:00.039) 0:01:33.703 ******* 2020-11-20T13:53:14.711 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2020-11-20T13:53:14.712 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:14.745 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:14 +0000 (0:00:00.040) 0:01:33.744 ******* 2020-11-20T13:53:14.752 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2020-11-20T13:53:14.753 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:14.786 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:14 +0000 (0:00:00.040) 0:01:33.784 ******* 2020-11-20T13:53:14.791 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-11-20T13:53:14.793 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:14.828 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:14 +0000 (0:00:00.042) 0:01:33.826 ******* 2020-11-20T13:53:14.833 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2020-11-20T13:53:14.835 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:14.868 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:14 +0000 (0:00:00.040) 0:01:33.866 ******* 2020-11-20T13:53:14.873 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-11-20T13:53:14.874 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:14.906 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:14 +0000 (0:00:00.038) 0:01:33.904 ******* 2020-11-20T13:53:14.911 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2020-11-20T13:53:14.912 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:14.945 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:14 +0000 (0:00:00.038) 0:01:33.943 ******* 2020-11-20T13:53:14.950 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2020-11-20T13:53:14.951 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:14.984 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:14 +0000 (0:00:00.039) 0:01:33.982 ******* 2020-11-20T13:53:14.989 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2020-11-20T13:53:14.990 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:15.023 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:15 +0000 (0:00:00.038) 0:01:34.021 ******* 2020-11-20T13:53:15.028 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2020-11-20T13:53:15.029 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:15.061 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:15 +0000 (0:00:00.038) 0:01:34.060 ******* 2020-11-20T13:53:15.066 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-11-20T13:53:15.068 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:15.100 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:15 +0000 (0:00:00.038) 0:01:34.099 ******* 2020-11-20T13:53:15.105 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2020-11-20T13:53:15.107 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:15.140 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:15 +0000 (0:00:00.039) 0:01:34.139 ******* 2020-11-20T13:53:15.145 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-11-20T13:53:15.146 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:15.180 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:15 +0000 (0:00:00.039) 0:01:34.178 ******* 2020-11-20T13:53:15.185 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2020-11-20T13:53:15.187 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:15.219 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:15 +0000 (0:00:00.039) 0:01:34.217 ******* 2020-11-20T13:53:15.223 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-11-20T13:53:15.225 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:15.260 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:15 +0000 (0:00:00.040) 0:01:34.258 ******* 2020-11-20T13:53:15.265 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2020-11-20T13:53:15.267 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:15.298 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:15 +0000 (0:00:00.038) 0:01:34.297 ******* 2020-11-20T13:53:15.303 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-11-20T13:53:15.305 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:15.338 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:15 +0000 (0:00:00.039) 0:01:34.336 ******* 2020-11-20T13:53:15.343 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2020-11-20T13:53:15.345 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:15.377 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:15 +0000 (0:00:00.039) 0:01:34.376 ******* 2020-11-20T13:53:15.382 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-11-20T13:53:15.384 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:15.416 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:15 +0000 (0:00:00.038) 0:01:34.414 ******* 2020-11-20T13:53:15.421 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2020-11-20T13:53:15.423 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:15.454 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:15 +0000 (0:00:00.038) 0:01:34.453 ******* 2020-11-20T13:53:15.459 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-11-20T13:53:15.461 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:15.493 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:15 +0000 (0:00:00.038) 0:01:34.491 ******* 2020-11-20T13:53:15.500 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2020-11-20T13:53:15.501 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:15.533 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:15 +0000 (0:00:00.039) 0:01:34.531 ******* 2020-11-20T13:53:15.538 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-11-20T13:53:15.540 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:15.572 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:15 +0000 (0:00:00.039) 0:01:34.570 ******* 2020-11-20T13:53:15.578 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2020-11-20T13:53:15.579 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:15.611 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:15 +0000 (0:00:00.039) 0:01:34.609 ******* 2020-11-20T13:53:15.616 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-11-20T13:53:15.617 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:15.650 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:15 +0000 (0:00:00.039) 0:01:34.648 ******* 2020-11-20T13:53:15.656 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-20T13:53:15.657 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2020-11-20T13:53:15.689 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:15 +0000 (0:00:00.039) 0:01:34.688 ******* 2020-11-20T13:53:15.694 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-20T13:53:15.696 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:15.729 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:15 +0000 (0:00:00.040) 0:01:34.728 ******* 2020-11-20T13:53:15.735 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2020-11-20T13:53:15.737 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2020-11-20T13:53:15.747 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2020-11-20T13:53:15.749 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2020-11-20T13:53:15.758 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-11-20T13:53:15.760 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi116.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2020-11-20T13:53:15.792 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:15 +0000 (0:00:00.062) 0:01:34.790 ******* 2020-11-20T13:53:15.797 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2020-11-20T13:53:15.799 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2020-11-20T13:53:15.836 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:15 +0000 (0:00:00.043) 0:01:34.834 ******* 2020-11-20T13:53:15.840 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-20T13:53:15.842 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:15.875 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:15 +0000 (0:00:00.039) 0:01:34.873 ******* 2020-11-20T13:53:15.901 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:15.934 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:15 +0000 (0:00:00.058) 0:01:34.932 ******* 2020-11-20T13:53:15.960 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-11-20T13:53:15.962 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:15.993 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:15 +0000 (0:00:00.058) 0:01:34.991 ******* 2020-11-20T13:53:16.018 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2020-11-20T13:53:16.020 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.002784", "end": "2020-11-20 13:53:15.991873", "rc": 0, "start": "2020-11-20 13:53:15.989089", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2020-11-20T13:53:16.056 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:16 +0000 (0:00:00.063) 0:01:35.055 ******* 2020-11-20T13:53:16.082 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2020-11-20T13:53:16.084 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:16.117 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:16 +0000 (0:00:00.060) 0:01:35.116 ******* 2020-11-20T13:53:16.153 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2020-11-20T13:53:16.155 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:16.185 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:16 +0000 (0:00:00.068) 0:01:35.184 ******* 2020-11-20T13:53:16.211 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2020-11-20T13:53:16.212 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Make raid/smart scripts work.] ******************************** ok: [smithi040.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/bin/lspci", "gid": 0, "group": "root", "mode": "0777", "owner": "root", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 11, "src": "/sbin/lspci", "state": "link", "uid": 0} 2020-11-20T13:53:16.244 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:16 +0000 (0:00:00.058) 0:01:35.243 ******* 2020-11-20T13:53:16.274 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:16 +0000 (0:00:00.029) 0:01:35.273 ******* 2020-11-20T13:53:16.300 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2020-11-20T13:53:16.302 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:16.334 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:16 +0000 (0:00:00.059) 0:01:35.333 ******* 2020-11-20T13:53:16.454 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-20T13:53:16.456 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/ltp/testcases/bin", "secontext": "unconfined_u:object_r:lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-11-20T13:53:16.488 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:16 +0000 (0:00:00.153) 0:01:35.486 ******* 2020-11-20T13:53:16.691 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-20T13:53:16.693 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/lib/ltp/testcases/bin/fsstress", "src": "/usr/bin/fsstress"} 2020-11-20T13:53:16.744 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:16 +0000 (0:00:00.255) 0:01:35.742 ******* 2020-11-20T13:53:16.770 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-11-20T13:53:16.772 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:16.803 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:16 +0000 (0:00:00.059) 0:01:35.802 ******* 2020-11-20T13:53:16.830 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-11-20T13:53:16.832 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:16.864 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:16 +0000 (0:00:00.060) 0:01:35.862 ******* 2020-11-20T13:53:16.890 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:16.923 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:16 +0000 (0:00:00.059) 0:01:35.922 ******* 2020-11-20T13:53:16.950 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-20T13:53:16.952 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:16.983 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:16 +0000 (0:00:00.059) 0:01:35.981 ******* 2020-11-20T13:53:17.008 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-20T13:53:17.010 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:17.042 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:17 +0000 (0:00:00.059) 0:01:36.040 ******* 2020-11-20T13:53:17.067 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-20T13:53:17.069 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:17.105 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:17 +0000 (0:00:00.063) 0:01:36.104 ******* 2020-11-20T13:53:17.132 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-20T13:53:17.134 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:17.166 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:17 +0000 (0:00:00.060) 0:01:36.164 ******* 2020-11-20T13:53:17.192 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:17.226 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:17 +0000 (0:00:00.060) 0:01:36.224 ******* 2020-11-20T13:53:17.242 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2020-11-20T13:53:17.244 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: gdisk", "Installed: device-mapper"]} 2020-11-20T13:53:17.278 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:17 +0000 (0:00:00.051) 0:01:36.276 ******* 2020-11-20T13:53:17.283 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-20T13:53:17.285 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:17.323 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:17 +0000 (0:00:00.045) 0:01:36.321 ******* 2020-11-20T13:53:17.328 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-11-20T13:53:17.330 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930529267712, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227179997, 'block_used': 12915111, 'inode_total': 61054976, 'inode_available': 60990124, 'inode_used': 64852, 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5'}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 227179997, "block_size": 4096, "block_total": 240095108, "block_used": 12915111, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60990124, "inode_total": 61054976, "inode_used": 64852, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930529267712, "size_total": 983429562368, "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}} 2020-11-20T13:53:17.362 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:17 +0000 (0:00:00.039) 0:01:36.361 ******* 2020-11-20T13:53:17.367 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2020-11-20T13:53:17.369 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:17.402 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:17 +0000 (0:00:00.040) 0:01:36.401 ******* 2020-11-20T13:53:17.429 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2020-11-20T13:53:17.431 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:17.466 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:17 +0000 (0:00:00.062) 0:01:36.464 ******* 2020-11-20T13:53:17.494 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-11-20T13:53:17.496 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:17.530 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:17 +0000 (0:00:00.064) 0:01:36.528 ******* 2020-11-20T13:53:17.547 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2020-11-20T13:53:17.548 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.005013", "end": "2020-11-20 13:53:17.530043", "rc": 0, "start": "2020-11-20 13:53:17.525030", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T13:53:17.580 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:17 +0000 (0:00:00.050) 0:01:36.579 ******* 2020-11-20T13:53:17.586 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-11-20T13:53:17.588 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:17.620 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:17 +0000 (0:00:00.040) 0:01:36.619 ******* 2020-11-20T13:53:17.626 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2020-11-20T13:53:17.627 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930529267712, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227179997, 'block_used': 12915111, 'inode_total': 61054976, 'inode_available': 60990124, 'inode_used': 64852, 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227179997, "block_size": 4096, "block_total": 240095108, "block_used": 12915111, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60990124, "inode_total": 61054976, "inode_used": 64852, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930529267712, "size_total": 983429562368, "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:17.658 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:17 +0000 (0:00:00.037) 0:01:36.657 ******* 2020-11-20T13:53:17.663 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2020-11-20T13:53:17.665 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:17.694 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:17 +0000 (0:00:00.035) 0:01:36.693 ******* 2020-11-20T13:53:17.699 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2020-11-20T13:53:17.733 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:17 +0000 (0:00:00.038) 0:01:36.731 ******* 2020-11-20T13:53:17.738 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2020-11-20T13:53:17.739 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:17.771 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:17 +0000 (0:00:00.038) 0:01:36.770 ******* 2020-11-20T13:53:17.797 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2020-11-20T13:53:17.799 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:17.828 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:17 +0000 (0:00:00.056) 0:01:36.826 ******* 2020-11-20T13:53:17.854 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2020-11-20T13:53:17.855 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:17.885 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:17 +0000 (0:00:00.056) 0:01:36.883 ******* 2020-11-20T13:53:17.910 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2020-11-20T13:53:17.912 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:17.948 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:17 +0000 (0:00:00.062) 0:01:36.946 ******* 2020-11-20T13:53:17.973 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2020-11-20T13:53:17.975 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:18.005 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:18 +0000 (0:00:00.057) 0:01:37.003 ******* 2020-11-20T13:53:18.031 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2020-11-20T13:53:18.033 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:18.065 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:18 +0000 (0:00:00.060) 0:01:37.064 ******* 2020-11-20T13:53:18.092 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2020-11-20T13:53:18.093 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:18.125 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:18 +0000 (0:00:00.059) 0:01:37.123 ******* 2020-11-20T13:53:18.150 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2020-11-20T13:53:18.152 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:18.184 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:18 +0000 (0:00:00.059) 0:01:37.182 ******* 2020-11-20T13:53:18.210 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-20T13:53:18.212 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2020-11-20T13:53:18.249 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:18 +0000 (0:00:00.064) 0:01:37.247 ******* 2020-11-20T13:53:18.286 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2020-11-20T13:53:18.288 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2020-11-20T13:53:18.297 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2020-11-20T13:53:18.308 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2020-11-20T13:53:18.309 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2020-11-20T13:53:18.319 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2020-11-20T13:53:18.320 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2020-11-20T13:53:18.330 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2020-11-20T13:53:18.390 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:18 +0000 (0:00:00.140) 0:01:37.388 ******* 2020-11-20T13:53:18.562 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2020-11-20T13:53:18.564 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.002844", "end": "2020-11-20 13:53:18.537841", "rc": 0, "start": "2020-11-20 13:53:18.534997", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2020-11-20T13:53:18.598 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:18 +0000 (0:00:00.208) 0:01:37.597 ******* 2020-11-20T13:53:18.624 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2020-11-20T13:53:18.626 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:18.660 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:18 +0000 (0:00:00.061) 0:01:37.658 ******* 2020-11-20T13:53:18.686 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2020-11-20T13:53:18.687 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:18.718 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:18 +0000 (0:00:00.058) 0:01:37.717 ******* 2020-11-20T13:53:18.744 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2020-11-20T13:53:18.746 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:18.777 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:18 +0000 (0:00:00.058) 0:01:37.775 ******* 2020-11-20T13:53:18.802 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2020-11-20T13:53:18.804 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:18.834 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:18 +0000 (0:00:00.057) 0:01:37.833 ******* 2020-11-20T13:53:19.323 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-20T13:53:19.325 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-3ugoQiC6cTGh2wh3bfBXbqsxKMiVJXhgUeLyWlD1JxEXM3z40cMet7rkL02MfQho'], 'uuids': ['6516f300-7a7d-499b-aac5-99e6e38748f4'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.375617", "end": "2020-11-20 13:53:19.293710", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-3ugoQiC6cTGh2wh3bfBXbqsxKMiVJXhgUeLyWlD1JxEXM3z40cMet7rkL02MfQho"], "labels": [], "masters": [], "uuids": ["6516f300-7a7d-499b-aac5-99e6e38748f4"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:53:17.918093", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T13:53:19.832 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2020-11-20T13:53:19.834 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: gdisk", "Installed: device-mapper"]} 2020-11-20T13:53:19.869 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:19 +0000 (0:00:01.034) 0:01:38.867 ******* 2020-11-20T13:53:19.906 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-11-20T13:53:19.908 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930529214464, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227179984, 'block_used': 12915124, 'inode_total': 61054976, 'inode_available': 60990120, 'inode_used': 64856, 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5'}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 227179984, "block_size": 4096, "block_total": 240095108, "block_used": 12915124, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60990120, "inode_total": 61054976, "inode_used": 64856, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930529214464, "size_total": 983429562368, "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}} 2020-11-20T13:53:19.941 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:19 +0000 (0:00:00.071) 0:01:38.939 ******* 2020-11-20T13:53:20.145 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2020-11-20T13:53:20.147 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.005167", "end": "2020-11-20 13:53:20.122940", "rc": 0, "start": "2020-11-20 13:53:20.117773", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T13:53:20.182 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:20 +0000 (0:00:00.241) 0:01:39.181 ******* 2020-11-20T13:53:20.219 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2020-11-20T13:53:20.221 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930529214464, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227179984, 'block_used': 12915124, 'inode_total': 61054976, 'inode_available': 60990120, 'inode_used': 64856, 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227179984, "block_size": 4096, "block_total": 240095108, "block_used": 12915124, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60990120, "inode_total": 61054976, "inode_used": 64856, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930529214464, "size_total": 983429562368, "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:20.253 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:20 +0000 (0:00:00.070) 0:01:39.252 ******* 2020-11-20T13:53:20.279 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2020-11-20T13:53:20.313 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:20 +0000 (0:00:00.059) 0:01:39.311 ******* 2020-11-20T13:53:20.610 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-20T13:53:20.612 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'virtual': 1, 'links': {'ids': ['lvm-pv-uuid-RUyFXR-mYFA-jJ08-ZXVr-1PBK-NrnR-dCkeMi', 'nvme-INTEL_SSDPEDMD400G4_PHFT620400VJ400BGN', 'nvme-nvme.8086-50484654363230343030564a34303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': [], 'labels': [], 'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4']}, 'vendor': None, 'model': 'INTEL SSDPEDMD400G4', 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': 'none', 'sectors': '781422768', 'sectorsize': '512', 'size': '372.61 GB', 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4']}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.092690", "end": "2020-11-20 13:53:20.582629", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-RUyFXR-mYFA-jJ08-ZXVr-1PBK-NrnR-dCkeMi", "nvme-INTEL_SSDPEDMD400G4_PHFT620400VJ400BGN", "nvme-nvme.8086-50484654363230343030564a34303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:53:19.489939", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! 2020-11-20T13:53:20.613 INFO:teuthology.task.ansible.out:You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T13:53:21.854 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-3ugoQiC6cTGh2wh3bfBXbqsxKMiVJXhgalEG6kYeXn8vdQmYvVPH9n2qX8NnBzjz'], 'uuids': ['32dec27a-0880-4623-9d54-1bb86df8dd40'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '31252480', 'sectorsize': '512', 'size': '14.90 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.053330", "end": "2020-11-20 13:53:21.825553", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-3ugoQiC6cTGh2wh3bfBXbqsxKMiVJXhgalEG6kYeXn8vdQmYvVPH9n2qX8NnBzjz"], "labels": [], "masters": [], "uuids": ["32dec27a-0880-4623-9d54-1bb86df8dd40"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:53:20.772223", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T13:53:21.938 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-LnOQVw5U8O7TT7aAr4lHBQFU4UBhNPnGlwBCZXaf7MXI5g5mm7qc3BKpnbRLxERU'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.393278", "end": "2020-11-20 13:53:21.903568", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-LnOQVw5U8O7TT7aAr4lHBQFU4UBhNPnGlwBCZXaf7MXI5g5mm7qc3BKpnbRLxERU"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:53:20.510290", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T13:53:23.105 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-3ugoQiC6cTGh2wh3bfBXbqsxKMiVJXhgIPYb55bSZLV4KJaZbwQ1M6KLXr34N4SG'], 'uuids': ['00e35a42-3c28-4d14-920f-59e883710555'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.063832", "end": "2020-11-20 13:53:23.076914", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-3ugoQiC6cTGh2wh3bfBXbqsxKMiVJXhgIPYb55bSZLV4KJaZbwQ1M6KLXr34N4SG"], "labels": [], "masters": [], "uuids": ["00e35a42-3c28-4d14-920f-59e883710555"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:53:22.013082", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T13:53:23.241 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'virtual': 1, 'links': {'ids': ['lvm-pv-uuid-RPJUon-RJ4g-JEma-Tjzx-VOPh-7HII-Ph8tGB', 'nvme-INTEL_SSDPEDMD400G4_CVFT5330009J400BGN', 'nvme-nvme.8086-43564654353333303030394a34303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': [], 'labels': [], 'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4']}, 'vendor': None, 'model': 'INTEL SSDPEDMD400G4', 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': 'none', 'sectors': '781422768', 'sectorsize': '512', 'size': '372.61 GB', 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4']}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.101082", "end": "2020-11-20 13:53:23.212269", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-RPJUon-RJ4g-JEma-Tjzx-VOPh-7HII-Ph8tGB", "nvme-INTEL_SSDPEDMD400G4_CVFT5330009J400BGN", "nvme-nvme.8086-43564654353333303030394a34303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:53:22.111187", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! 2020-11-20T13:53:23.243 INFO:teuthology.task.ansible.out:You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T13:53:24.348 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-3ugoQiC6cTGh2wh3bfBXbqsxKMiVJXhgrxoHciPYbaCd0LSHklcxXnm0EMXUrrUr'], 'uuids': ['ed3ec385-caa5-4bea-a9f4-6aed4b01ba73'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.056097", "end": "2020-11-20 13:53:24.317391", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-3ugoQiC6cTGh2wh3bfBXbqsxKMiVJXhgrxoHciPYbaCd0LSHklcxXnm0EMXUrrUr"], "labels": [], "masters": [], "uuids": ["ed3ec385-caa5-4bea-a9f4-6aed4b01ba73"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:53:23.261294", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T13:53:24.350 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'virtual': 1, 'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QJB5', 'wwn-0x5000c50091e1a66f'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': 'ATA', 'model': 'ST1000NM0033-9ZM', 'sas_address': None, 'sas_device_handle': None, 'serial': 'Z1W5QJB5', 'removable': '0', 'support_discard': '0', 'wwn': '0x5000c50091e1a66f', 'partitions': {'sda1': {'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QJB5-part1', 'wwn-0x5000c50091e1a66f-part1'], 'uuids': ['9bdf28e7-5b1a-4776-884c-191bd8ad39f5'], 'labels': [], 'masters': []}, 'start': '2048', 'sectors': '1953522688', 'sectorsize': 512, 'size': '931.51 GB', 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5', 'holders': []}}, 'rotational': '1', 'scheduler_mode': 'mq-deadline', 'sectors': '1953525168', 'sectorsize': '512', 'size': '931.51 GB', 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'holders': []}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QJB5", "wwn-0x5000c50091e1a66f"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QJB5-part1", "wwn-0x5000c50091e1a66f-part1"], "labels": [], "masters": [], "uuids": ["9bdf28e7-5b1a-4776-884c-191bd8ad39f5"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5QJB5", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50091e1a66f"}}, 2020-11-20T13:53:24.351 INFO:teuthology.task.ansible.out: "skip_reason": "Conditional result was False"} 2020-11-20T13:53:24.527 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-LnOQVw5U8O7TT7aAr4lHBQFU4UBhNPnGi4QVFnkOoDdfbDHmHHACCf66P6zHIDJn'], 'uuids': ['09b1b0f4-5659-441a-9b7e-d0939ab7ac00'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '31252480', 'sectorsize': '512', 'size': '14.90 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.081724", "end": "2020-11-20 13:53:24.492619", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-LnOQVw5U8O7TT7aAr4lHBQFU4UBhNPnGi4QVFnkOoDdfbDHmHHACCf66P6zHIDJn"], "labels": [], "masters": [], "uuids": ["09b1b0f4-5659-441a-9b7e-d0939ab7ac00"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:53:23.410895", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T13:53:25.607 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-3ugoQiC6cTGh2wh3bfBXbqsxKMiVJXhgcfdVoVSiVdiOIbJlcJHsqcR4fICK3MWc'], 'uuids': ['ecca3776-16cb-436a-bfcb-93e38efb70b6'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.066315", "end": "2020-11-20 13:53:25.578255", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-3ugoQiC6cTGh2wh3bfBXbqsxKMiVJXhgcfdVoVSiVdiOIbJlcJHsqcR4fICK3MWc"], "labels": [], "masters": [], "uuids": ["ecca3776-16cb-436a-bfcb-93e38efb70b6"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:53:24.511940", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T13:53:25.645 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:25 +0000 (0:00:05.332) 0:01:44.644 ******* 2020-11-20T13:53:25.776 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-LnOQVw5U8O7TT7aAr4lHBQFU4UBhNPnG2tghLVXaPk4PlA5l5w8wXEDUokUdCFQu'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.048947", "end": "2020-11-20 13:53:25.743505", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-LnOQVw5U8O7TT7aAr4lHBQFU4UBhNPnG2tghLVXaPk4PlA5l5w8wXEDUokUdCFQu"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:53:24.694558", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T13:53:25.995 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-20T13:53:25.996 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-3ugoQiC6cTGh2wh3bfBXbqsxKMiVJXhgUeLyWlD1JxEXM3z40cMet7rkL02MfQho'], 'uuids': ['6516f300-7a7d-499b-aac5-99e6e38748f4'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.136558", "end": "2020-11-20 13:53:25.971716", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-3ugoQiC6cTGh2wh3bfBXbqsxKMiVJXhgUeLyWlD1JxEXM3z40cMet7rkL02MfQho"], "labels": [], "masters": [], "uuids": ["6516f300-7a7d-499b-aac5-99e6e38748f4"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:53:25.835158", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.134115 s, 860 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.134115 s, 860 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T13:53:26.214 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'virtual': 1, 'links': {'ids': ['lvm-pv-uuid-RUyFXR-mYFA-jJ08-ZXVr-1PBK-NrnR-dCkeMi', 'nvme-INTEL_SSDPEDMD400G4_PHFT620400VJ400BGN', 'nvme-nvme.8086-50484654363230343030564a34303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': [], 'labels': [], 'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4']}, 'vendor': None, 'model': 'INTEL SSDPEDMD400G4', 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': 'none', 'sectors': '781422768', 'sectorsize': '512', 'size': '372.61 GB', 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4']}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.036893", "end": "2020-11-20 13:53:26.184324", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-RUyFXR-mYFA-jJ08-ZXVr-1PBK-NrnR-dCkeMi", "nvme-INTEL_SSDPEDMD400G4_PHFT620400VJ400BGN", "nvme-nvme.8086-50484654363230343030564a34303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:53:26.147431", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0343591 s, 3.4 GB/s", "std 2020-11-20T13:53:26.216 INFO:teuthology.task.ansible.out:err_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0343591 s, 3.4 GB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T13:53:26.535 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-3ugoQiC6cTGh2wh3bfBXbqsxKMiVJXhgalEG6kYeXn8vdQmYvVPH9n2qX8NnBzjz'], 'uuids': ['32dec27a-0880-4623-9d54-1bb86df8dd40'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '31252480', 'sectorsize': '512', 'size': '14.90 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.139829", "end": "2020-11-20 13:53:26.509150", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-3ugoQiC6cTGh2wh3bfBXbqsxKMiVJXhgalEG6kYeXn8vdQmYvVPH9n2qX8NnBzjz"], "labels": [], "masters": [], "uuids": ["32dec27a-0880-4623-9d54-1bb86df8dd40"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:53:26.369321", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.137347 s, 840 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.137347 s, 840 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T13:53:26.847 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-3ugoQiC6cTGh2wh3bfBXbqsxKMiVJXhgIPYb55bSZLV4KJaZbwQ1M6KLXr34N4SG'], 'uuids': ['00e35a42-3c28-4d14-920f-59e883710555'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.138940", "end": "2020-11-20 13:53:26.824820", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-3ugoQiC6cTGh2wh3bfBXbqsxKMiVJXhgIPYb55bSZLV4KJaZbwQ1M6KLXr34N4SG"], "labels": [], "masters": [], "uuids": ["00e35a42-3c28-4d14-920f-59e883710555"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:53:26.685880", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.136424 s, 845 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.136424 s, 845 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T13:53:27.022 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-20T13:53:27.024 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-LnOQVw5U8O7TT7aAr4lHBQFU4UBhNPnGx05LYfSDddROt6QLrU3sk1On2zFPY60E'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.049208", "end": "2020-11-20 13:53:26.993655", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-LnOQVw5U8O7TT7aAr4lHBQFU4UBhNPnGx05LYfSDddROt6QLrU3sk1On2zFPY60E"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:53:25.944447", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T13:53:27.034 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'virtual': 1, 'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W4HQ07', 'wwn-0x5000c500871a8a38'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': 'ATA', 'model': 'ST1000NM0033-9ZM', 'sas_address': None, 'sas_device_handle': None, 'serial': 'Z1W4HQ07', 'removable': '0', 'support_discard': '0', 'wwn': '0x5000c500871a8a38', 'partitions': {'sda1': {'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W4HQ07-part1', 'wwn-0x5000c500871a8a38-part1'], 'uuids': ['9bdf28e7-5b1a-4776-884c-191bd8ad39f5'], 'labels': [], 'masters': []}, 'start': '2048', 'sectors': '1953522688', 'sectorsize': 512, 'size': '931.51 GB', 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5', 'holders': []}}, 'rotational': '1', 'scheduler_mode': 'mq-deadline', 'sectors': '1953525168', 'sectorsize': '512', 'size': '931.51 GB', 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'holders': []}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4HQ07", "wwn-0x5000c500871a8a38"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4HQ07-part1", "wwn-0x5000c500871a8a38-part1"], "labels": [], "masters": [], "uuids": ["9bdf28e7-5b1a-4776-884c-191bd8ad39f5"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4HQ07", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500871a8a38"}}, 2020-11-20T13:53:27.036 INFO:teuthology.task.ansible.out: "skip_reason": "Conditional result was False"} 2020-11-20T13:53:27.181 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-20T13:53:27.182 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-3ugoQiC6cTGh2wh3bfBXbqsxKMiVJXhgrxoHciPYbaCd0LSHklcxXnm0EMXUrrUr'], 'uuids': ['ed3ec385-caa5-4bea-a9f4-6aed4b01ba73'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.149671", "end": "2020-11-20 13:53:27.153178", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-3ugoQiC6cTGh2wh3bfBXbqsxKMiVJXhgrxoHciPYbaCd0LSHklcxXnm0EMXUrrUr"], "labels": [], "masters": [], "uuids": ["ed3ec385-caa5-4bea-a9f4-6aed4b01ba73"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:53:27.003507", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.147152 s, 784 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.147152 s, 784 MB/s"], "stdout": "", "stdout_lines": []} skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'virtual': 1, 'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QJB5', 'wwn-0x5000c50091e1a66f'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': 'ATA', 'model': 'ST1000NM0033-9ZM', 'sas_address': None, 'sas_device_handle': None, 'serial': 'Z1W5QJB5', 'removable': '0', 'support_discard': '0', 'wwn': '0x5000c50091e1a66f', 'partition 2020-11-20T13:53:27.184 INFO:teuthology.task.ansible.out:s': {'sda1': {'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QJB5-part1', 'wwn-0x5000c50091e1a66f-part1'], 'uuids': ['9bdf28e7-5b1a-4776-884c-191bd8ad39f5'], 'labels': [], 'masters': []}, 'start': '2048', 'sectors': '1953522688', 'sectorsize': 512, 'size': '931.51 GB', 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5', 'holders': []}}, 'rotational': '1', 'scheduler_mode': 'mq-deadline', 'sectors': '1953525168', 'sectorsize': '512', 'size': '931.51 GB', 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'holders': []}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QJB5", "wwn-0x5000c50091e1a66f"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QJB5-part1", "wwn-0x5000c50091e1a66f-part1"], "labels": [], "masters": [], "uuids": ["9bdf28e7-5b1a-4776-884c-191bd8ad39f5"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5QJB5", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50091e1a66f"}}, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:27.494 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-3ugoQiC6cTGh2wh3bfBXbqsxKMiVJXhgcfdVoVSiVdiOIbJlcJHsqcR4fICK3MWc'], 'uuids': ['ecca3776-16cb-436a-bfcb-93e38efb70b6'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.137053", "end": "2020-11-20 13:53:27.473367", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-3ugoQiC6cTGh2wh3bfBXbqsxKMiVJXhgcfdVoVSiVdiOIbJlcJHsqcR4fICK3MWc"], "labels": [], "masters": [], "uuids": ["ecca3776-16cb-436a-bfcb-93e38efb70b6"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:53:27.336314", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.134657 s, 857 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.134657 s, 857 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T13:53:27.532 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:27 +0000 (0:00:01.886) 0:01:46.530 ******* 2020-11-20T13:53:27.796 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2020-11-20T13:53:27.797 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.072692", "end": "2020-11-20 13:53:27.765659", "rc": 0, "start": "2020-11-20 13:53:27.692967", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T13:53:27.828 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:27 +0000 (0:00:00.296) 0:01:46.827 ******* 2020-11-20T13:53:28.061 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2020-11-20T13:53:28.062 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.051483", "end": "2020-11-20 13:53:28.041267", "rc": 0, "start": "2020-11-20 13:53:27.989784", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T13:53:28.094 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:28 +0000 (0:00:00.265) 0:01:47.092 ******* 2020-11-20T13:53:28.120 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2020-11-20T13:53:28.156 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:28 +0000 (0:00:00.062) 0:01:47.154 ******* 2020-11-20T13:53:28.181 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2020-11-20T13:53:28.183 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:28.217 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:28 +0000 (0:00:00.060) 0:01:47.215 ******* 2020-11-20T13:53:28.243 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2020-11-20T13:53:28.244 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:28.276 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:28 +0000 (0:00:00.059) 0:01:47.274 ******* 2020-11-20T13:53:28.282 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-20T13:53:28.287 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-LnOQVw5U8O7TT7aAr4lHBQFU4UBhNPnGzHjY6XEmXiXkcjTeAWVzJ1lkZUkx0NAT'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.049930", "end": "2020-11-20 13:53:28.252078", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-LnOQVw5U8O7TT7aAr4lHBQFU4UBhNPnGzHjY6XEmXiXkcjTeAWVzJ1lkZUkx0NAT"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:53:27.202148", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T13:53:28.325 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:28 +0000 (0:00:00.049) 0:01:47.324 ******* 2020-11-20T13:53:28.331 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-11-20T13:53:28.333 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:28.368 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:28 +0000 (0:00:00.042) 0:01:47.367 ******* 2020-11-20T13:53:28.393 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2020-11-20T13:53:28.395 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:28.426 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:28 +0000 (0:00:00.057) 0:01:47.424 ******* 2020-11-20T13:53:28.609 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2020-11-20T13:53:28.611 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-20T13:53:28.642 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:28 +0000 (0:00:00.215) 0:01:47.640 ******* 2020-11-20T13:53:28.685 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-11-20T13:53:28.687 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930529267712, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227179997, 'block_used': 12915111, 'inode_total': 61054976, 'inode_available': 60990124, 'inode_used': 64852, 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227179997, "block_size": 4096, "block_total": 240095108, "block_used": 12915111, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60990124, "inode_total": 61054976, "inode_used": 64852, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930529267712, "size_total": 983429562368, "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}, "skip_reason": "Conditional result was False"} TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-20T13:53:28.687 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-LnOQVw5U8O7TT7aAr4lHBQFU4UBhNPnGlwBCZXaf7MXI5g5mm7qc3BKpnbRLxERU'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.134470", "end": "2020-11-20 13:53:28.663308", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-LnOQVw5U8O7TT7aAr4lHBQFU4UBhNPnGlwBCZXaf7MXI5g5mm7qc3BKpnbRLxERU"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_addr 2020-11-20T13:53:28.689 INFO:teuthology.task.ansible.out:ess": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:53:28.528838", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.13176 s, 875 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.13176 s, 875 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T13:53:28.720 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:28 +0000 (0:00:00.078) 0:01:47.719 ******* 2020-11-20T13:53:28.746 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2020-11-20T13:53:28.748 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:28.779 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:28 +0000 (0:00:00.058) 0:01:47.778 ******* 2020-11-20T13:53:28.806 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:28.837 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:28 +0000 (0:00:00.057) 0:01:47.836 ******* 2020-11-20T13:53:28.862 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-20T13:53:28.864 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:28.894 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:28 +0000 (0:00:00.056) 0:01:47.892 ******* 2020-11-20T13:53:28.921 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-20T13:53:28.923 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'virtual': 1, 'links': {'ids': ['lvm-pv-uuid-RPJUon-RJ4g-JEma-Tjzx-VOPh-7HII-Ph8tGB', 'nvme-INTEL_SSDPEDMD400G4_CVFT5330009J400BGN', 'nvme-nvme.8086-43564654353333303030394a34303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': [], 'labels': [], 'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4']}, 'vendor': None, 'model': 'INTEL SSDPEDMD400G4', 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': 'none', 'sectors': '781422768', 'sectorsize': '512', 'size': '372.61 GB', 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4']}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.036092", "end": "2020-11-20 13:53:28.891142", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-RPJUon-RJ4g-JEma-Tjzx-VOPh-7HII-Ph8tGB", "nvme-INTEL_SSDPEDMD400G4_CVFT5330009J400BGN", "nvme-nvme.8086-43564654353333303030394a34303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:53:28.855050", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.033447 s, 3.4 GB/s", "stde 2020-11-20T13:53:28.925 INFO:teuthology.task.ansible.out:rr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.033447 s, 3.4 GB/s"], "stdout": "", "stdout_lines": []} TASK [testnode : Create volume_groups dict] ************************************ skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:28.956 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:28 +0000 (0:00:00.062) 0:01:47.954 ******* 2020-11-20T13:53:28.982 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2020-11-20T13:53:28.984 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:29.016 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:29 +0000 (0:00:00.060) 0:01:48.015 ******* 2020-11-20T13:53:29.042 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2020-11-20T13:53:29.044 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:29.075 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:29 +0000 (0:00:00.058) 0:01:48.073 ******* 2020-11-20T13:53:29.256 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-20T13:53:29.258 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-LnOQVw5U8O7TT7aAr4lHBQFU4UBhNPnGi4QVFnkOoDdfbDHmHHACCf66P6zHIDJn'], 'uuids': ['09b1b0f4-5659-441a-9b7e-d0939ab7ac00'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '31252480', 'sectorsize': '512', 'size': '14.90 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.136643", "end": "2020-11-20 13:53:29.227413", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-LnOQVw5U8O7TT7aAr4lHBQFU4UBhNPnGi4QVFnkOoDdfbDHmHHACCf66P6zHIDJn"], "labels": [], "masters": [], "uuids": ["09b1b0f4-5659-441a-9b7e-d0939ab7ac00"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:53:29.090770", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.134068 s, 860 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.134068 s, 860 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T13:53:29.579 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-LnOQVw5U8O7TT7aAr4lHBQFU4UBhNPnG2tghLVXaPk4PlA5l5w8wXEDUokUdCFQu'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.135239", "end": "2020-11-20 13:53:29.551376", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-LnOQVw5U8O7TT7aAr4lHBQFU4UBhNPnG2tghLVXaPk4PlA5l5w8wXEDUokUdCFQu"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:53:29.416137", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.132698 s, 869 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.132698 s, 869 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T13:53:29.746 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2020-11-20T13:53:29.747 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-11-20T13:53:29.779 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:29 +0000 (0:00:00.704) 0:01:48.778 ******* 2020-11-20T13:53:29.913 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-20T13:53:29.915 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-LnOQVw5U8O7TT7aAr4lHBQFU4UBhNPnGx05LYfSDddROt6QLrU3sk1On2zFPY60E'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.147493", "end": "2020-11-20 13:53:29.886067", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-LnOQVw5U8O7TT7aAr4lHBQFU4UBhNPnGx05LYfSDddROt6QLrU3sk1On2zFPY60E"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:53:29.738574", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.144903 s, 796 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.144903 s, 796 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T13:53:29.926 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'virtual': 1, 'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W4HQ07', 'wwn-0x5000c500871a8a38'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': 'ATA', 'model': 'ST1000NM0033-9ZM', 'sas_address': None, 'sas_device_handle': None, 'serial': 'Z1W4HQ07', 'removable': '0', 'support_discard': '0', 'wwn': '0x5000c500871a8a38', 'partitions': {'sda1': {'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W4HQ07-part1', 'wwn-0x5000c500871a8a38-part1'], 'uuids': ['9bdf28e7-5b1a-4776-884c-191bd8ad39f5'], 'labels': [], 'masters': []}, 'start': '2048', 'sectors': '1953522688', 'sectorsize': 512, 'size': '931.51 GB', 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5', 'holders': []}}, 'rotational': '1', 'scheduler_mode': 'mq-deadline', 'sectors': '1953525168', 'sectorsize': '512', 'size': '931.51 GB', 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'holders': []}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4HQ07", "wwn-0x5000c500871a8a38"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4HQ07-part1", "wwn-0x5000c500871a8a38-part1"], "labels": [], "masters": [], "uuids": ["9bdf28e7-5b1a-4776-884c-191bd8ad39f5"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4HQ07", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500871a8a38"}}, 2020-11-20T13:53:29.927 INFO:teuthology.task.ansible.out: "skip_reason": "Conditional result was False"} 2020-11-20T13:53:30.238 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-LnOQVw5U8O7TT7aAr4lHBQFU4UBhNPnGzHjY6XEmXiXkcjTeAWVzJ1lkZUkx0NAT'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.135326", "end": "2020-11-20 13:53:30.217069", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-LnOQVw5U8O7TT7aAr4lHBQFU4UBhNPnGzHjY6XEmXiXkcjTeAWVzJ1lkZUkx0NAT"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:53:30.081743", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.13268 s, 869 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.13268 s, 869 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T13:53:30.277 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:30 +0000 (0:00:00.497) 0:01:49.276 ******* 2020-11-20T13:53:30.418 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-11-20T13:53:30.420 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-20T13:53:30.554 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2020-11-20T13:53:30.556 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.072595", "end": "2020-11-20 13:53:30.523038", "rc": 0, "start": "2020-11-20 13:53:30.450443", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T13:53:30.587 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:30 +0000 (0:00:00.310) 0:01:49.586 ******* 2020-11-20T13:53:30.748 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-11-20T13:53:30.750 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-20T13:53:30.833 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2020-11-20T13:53:30.835 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.054650", "end": "2020-11-20 13:53:30.815880", "rc": 0, "start": "2020-11-20 13:53:30.761230", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T13:53:30.865 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:30 +0000 (0:00:00.277) 0:01:49.864 ******* 2020-11-20T13:53:30.891 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2020-11-20T13:53:30.927 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:30 +0000 (0:00:00.061) 0:01:49.925 ******* 2020-11-20T13:53:30.953 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2020-11-20T13:53:30.955 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:30.989 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:30 +0000 (0:00:00.062) 0:01:49.987 ******* 2020-11-20T13:53:31.015 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2020-11-20T13:53:31.017 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:31.048 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:31 +0000 (0:00:00.059) 0:01:50.046 ******* 2020-11-20T13:53:31.075 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-11-20T13:53:31.076 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:31.111 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:31 +0000 (0:00:00.062) 0:01:50.109 ******* 2020-11-20T13:53:31.117 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-11-20T13:53:31.119 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-20T13:53:31.139 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2020-11-20T13:53:31.141 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:31.173 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:31 +0000 (0:00:00.062) 0:01:50.171 ******* 2020-11-20T13:53:31.377 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2020-11-20T13:53:31.379 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-20T13:53:31.410 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:31 +0000 (0:00:00.237) 0:01:50.409 ******* 2020-11-20T13:53:31.416 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-11-20T13:53:31.418 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-20T13:53:31.449 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-11-20T13:53:31.450 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930529214464, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227179984, 'block_used': 12915124, 'inode_total': 61054976, 'inode_available': 60990120, 'inode_used': 64856, 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227179984, "block_size": 4096, "block_total": 240095108, "block_used": 12915124, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60990120, "inode_total": 61054976, "inode_used": 64856, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930529214464, "size_total": 983429562368, "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:31.501 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:31 +0000 (0:00:00.090) 0:01:50.499 ******* 2020-11-20T13:53:31.529 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:31.568 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:31 +0000 (0:00:00.066) 0:01:50.566 ******* 2020-11-20T13:53:31.594 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-20T13:53:31.596 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:31.627 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:31 +0000 (0:00:00.059) 0:01:50.625 ******* 2020-11-20T13:53:31.654 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:31.688 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:31 +0000 (0:00:00.061) 0:01:50.687 ******* 2020-11-20T13:53:31.714 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2020-11-20T13:53:31.716 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:31.748 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:31 +0000 (0:00:00.059) 0:01:50.746 ******* 2020-11-20T13:53:31.754 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-11-20T13:53:31.756 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-20T13:53:31.791 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:31 +0000 (0:00:00.043) 0:01:50.789 ******* 2020-11-20T13:53:31.796 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2020-11-20T13:53:31.798 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:31.832 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:31 +0000 (0:00:00.040) 0:01:50.830 ******* 2020-11-20T13:53:31.858 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2020-11-20T13:53:31.859 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:31.890 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:31 +0000 (0:00:00.058) 0:01:50.888 ******* 2020-11-20T13:53:31.979 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-11-20T13:53:31.981 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-11-20T13:53:32.015 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:32 +0000 (0:00:00.124) 0:01:51.013 ******* 2020-11-20T13:53:32.221 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-11-20T13:53:32.222 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-20T13:53:32.357 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** changed: [smithi040.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-11-20T13:53:32.391 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:32 +0000 (0:00:00.376) 0:01:51.390 ******* 2020-11-20T13:53:32.408 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-11-20T13:53:32.410 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-20T13:53:32.576 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-20T13:53:32.754 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-20T13:53:32.755 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* changed: [smithi040.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-20T13:53:32.765 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-11-20T13:53:32.767 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:32.802 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:32 +0000 (0:00:00.410) 0:01:51.800 ******* 2020-11-20T13:53:32.996 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2020-11-20T13:53:32.998 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-11-20T13:53:33.028 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:33 +0000 (0:00:00.226) 0:01:52.027 ******* 2020-11-20T13:53:33.066 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2020-11-20T13:53:33.067 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2020-11-20T13:53:33.105 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:33 +0000 (0:00:00.076) 0:01:52.104 ******* 2020-11-20T13:53:33.110 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-11-20T13:53:33.112 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-20T13:53:33.435 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-20T13:53:33.777 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-20T13:53:34.141 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-20T13:53:34.186 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:34 +0000 (0:00:01.080) 0:01:53.184 ******* 2020-11-20T13:53:34.388 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-11-20T13:53:34.390 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-11-20T13:53:34.420 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:34 +0000 (0:00:00.234) 0:01:53.419 ******* 2020-11-20T13:53:34.622 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2020-11-20T13:53:34.623 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => {"changed": true} 2020-11-20T13:53:34.656 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:34 +0000 (0:00:00.235) 0:01:53.655 ******* 2020-11-20T13:53:34.662 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-11-20T13:53:34.664 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-20T13:53:34.831 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-20T13:53:35.011 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-20T13:53:35.208 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-20T13:53:35.220 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:35.256 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:35 +0000 (0:00:00.599) 0:01:54.254 ******* 2020-11-20T13:53:35.262 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2020-11-20T13:53:35.264 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-11-20T13:53:35.297 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:35 +0000 (0:00:00.041) 0:01:54.296 ******* 2020-11-20T13:53:35.323 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2020-11-20T13:53:35.325 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:35.356 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:35 +0000 (0:00:00.059) 0:01:54.355 ******* 2020-11-20T13:53:35.383 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2020-11-20T13:53:35.385 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:35.416 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:35 +0000 (0:00:00.059) 0:01:54.415 ******* 2020-11-20T13:53:35.442 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2020-11-20T13:53:35.444 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:35.478 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:35 +0000 (0:00:00.061) 0:01:54.477 ******* 2020-11-20T13:53:35.484 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2020-11-20T13:53:35.486 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-11-20T13:53:35.520 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:35 +0000 (0:00:00.041) 0:01:54.519 ******* 2020-11-20T13:53:35.526 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2020-11-20T13:53:35.528 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:35.561 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:35 +0000 (0:00:00.040) 0:01:54.559 ******* 2020-11-20T13:53:35.566 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2020-11-20T13:53:35.567 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2020-11-20T13:53:35.599 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:35 +0000 (0:00:00.038) 0:01:54.597 ******* 2020-11-20T13:53:35.604 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2020-11-20T13:53:35.606 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:35.638 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:35 +0000 (0:00:00.038) 0:01:54.636 ******* 2020-11-20T13:53:35.664 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2020-11-20T13:53:35.665 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:35.695 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:35 +0000 (0:00:00.057) 0:01:54.694 ******* 2020-11-20T13:53:35.721 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2020-11-20T13:53:35.723 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:35.754 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:35 +0000 (0:00:00.058) 0:01:54.752 ******* 2020-11-20T13:53:35.779 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2020-11-20T13:53:35.781 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:35.812 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:35 +0000 (0:00:00.058) 0:01:54.810 ******* 2020-11-20T13:53:35.838 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2020-11-20T13:53:35.839 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:35.869 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:35 +0000 (0:00:00.057) 0:01:54.868 ******* 2020-11-20T13:53:35.894 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2020-11-20T13:53:35.896 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:35.926 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:35 +0000 (0:00:00.057) 0:01:54.925 ******* 2020-11-20T13:53:35.952 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2020-11-20T13:53:35.954 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:35.984 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:35 +0000 (0:00:00.057) 0:01:54.982 ******* 2020-11-20T13:53:36.424 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2020-11-20T13:53:36.425 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "checksum": "a7ecef00073feab34f868065d34e9dac55134241", "dest": "/etc/chrony.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/chrony.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 215, "state": "file", "uid": 0} 2020-11-20T13:53:36.457 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:36 +0000 (0:00:00.472) 0:01:55.455 ******* 2020-11-20T13:53:36.753 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2020-11-20T13:53:36.754 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "chronyd", "state": "started", "status": {"ActiveEnterTimestamp": "Fri 2020-11-20 13:50:49 UTC", "ActiveEnterTimestampMonotonic": "15528295", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "ntpdate.service system.slice systemd-journald.socket ntpd.service sysinit.target basic.target tmp.mount systemd-tmpfiles-setup.service -.mount sntp.service", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Fri 2020-11-20 13:50:48 UTC", "AssertTimestampMonotonic": "14443869", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Fri 2020-11-20 13:50:48 UTC", "ConditionTimestampMonotonic": "14443849", "ConfigurationDirectoryMode": "0755", "Conflicts": "systemd-timesyncd.service shutdown.target ntpd.service", "ControlGroup": "/system.slice/chronyd.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description" 2020-11-20T13:53:36.756 INFO:teuthology.task.ansible.out:: "NTP client/server", "DevicePolicy": "auto", "Documentation": "man:chronyd(8) man:chrony.conf(5)", "DynamicUser": "no", "EnvironmentFiles": "/etc/sysconfig/chronyd (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "2415", "ExecMainStartTimestamp": "Fri 2020-11-20 13:50:49 UTC", "ExecMainStartTimestampMonotonic": "15484013", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/sbin/chronyd ; argv[]=/usr/sbin/chronyd $OPTIONS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartPost": "{ path=/usr/libexec/chrony-helper ; argv[]=/usr/libexec/chrony-helper update-daemon ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/chronyd.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "chronyd.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Fri 2020-11-20 13:50:48 UTC", "InactiveExitTimestampMonotonic": "14486107", "InvocationID": "92946544c26f46adb4faf3273d00b71f", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "167772 2020-11-20T13:53:36.758 INFO:teuthology.task.ansible.out:16", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127426", "LimitNPROCSoft": "127426", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127426", "LimitSIGPENDINGSoft": "127426", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "2415", "MemoryAccounting": "yes", "MemoryCurrent": "2711552", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "chronyd.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PIDFile": "/run/chrony/chronyd.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "yes", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "full", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount sysinit.target system.slice", "RequiresMountsFor": "/var/tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardE 2020-11-20T13:53:36.759 INFO:teuthology.task.ansible.out:rror": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Fri 2020-11-20 13:50:49 UTC", "StateChangeTimestampMonotonic": "15528295", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Fri 2020-11-20 13:50:49 UTC", "WatchdogTimestampMonotonic": "15484026", "WatchdogUSec": "0"}} 2020-11-20T13:53:36.788 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:36 +0000 (0:00:00.330) 0:01:55.786 ******* 2020-11-20T13:53:36.814 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-20T13:53:36.816 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2020-11-20T13:53:36.852 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:36 +0000 (0:00:00.063) 0:01:55.850 ******* 2020-11-20T13:53:36.880 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-20T13:53:36.882 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"ansible_facts": {"ansible_python_interpreter": "/usr/bin/python3", "pip_executable": "pip3", "pip_version": "python3-pip"}, "changed": false} 2020-11-20T13:53:36.913 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:36 +0000 (0:00:00.061) 0:01:55.912 ******* 2020-11-20T13:53:36.918 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2020-11-20T13:53:36.920 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => {"changed": true} 2020-11-20T13:53:36.952 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:36 +0000 (0:00:00.038) 0:01:55.950 ******* 2020-11-20T13:53:36.957 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2020-11-20T13:53:36.958 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:36.991 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:36 +0000 (0:00:00.039) 0:01:55.990 ******* 2020-11-20T13:53:37.017 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2020-11-20T13:53:37.019 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:37.050 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:37 +0000 (0:00:00.058) 0:01:56.048 ******* 2020-11-20T13:53:37.076 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2020-11-20T13:53:37.078 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:37.108 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:37 +0000 (0:00:00.058) 0:01:56.106 ******* 2020-11-20T13:53:37.270 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2020-11-20T13:53:37.271 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-11-20T13:53:37.304 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:37 +0000 (0:00:00.195) 0:01:56.302 ******* 2020-11-20T13:53:37.310 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2020-11-20T13:53:37.311 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 4096, "state": "directory", "uid": 1000} 2020-11-20T13:53:37.343 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:37 +0000 (0:00:00.039) 0:01:56.342 ******* 2020-11-20T13:53:37.349 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2020-11-20T13:53:37.351 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:37.383 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:37 +0000 (0:00:00.039) 0:01:56.382 ******* 2020-11-20T13:53:37.409 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:37.440 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:37 +0000 (0:00:00.057) 0:01:56.439 ******* 2020-11-20T13:53:37.466 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2020-11-20T13:53:37.467 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:37.498 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:37 +0000 (0:00:00.057) 0:01:56.496 ******* 2020-11-20T13:53:37.524 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2020-11-20T13:53:37.526 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:37.558 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:37 +0000 (0:00:00.059) 0:01:56.556 ******* 2020-11-20T13:53:37.584 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2020-11-20T13:53:37.586 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:37.617 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:37 +0000 (0:00:00.059) 0:01:56.616 ******* 2020-11-20T13:53:37.643 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2020-11-20T13:53:37.645 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:37.676 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:37 +0000 (0:00:00.058) 0:01:56.674 ******* 2020-11-20T13:53:37.702 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2020-11-20T13:53:37.704 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:37.735 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:37 +0000 (0:00:00.058) 0:01:56.733 ******* 2020-11-20T13:53:37.761 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:37.794 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:37 +0000 (0:00:00.058) 0:01:56.792 ******* 2020-11-20T13:53:37.814 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2020-11-20T13:53:37.816 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 73, "state": "file", "uid": 1000} 2020-11-20T13:53:37.847 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:37 +0000 (0:00:00.052) 0:01:56.845 ******* 2020-11-20T13:53:37.853 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2020-11-20T13:53:37.854 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:37.888 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:37 +0000 (0:00:00.041) 0:01:56.886 ******* 2020-11-20T13:53:37.914 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2020-11-20T13:53:37.916 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:37.947 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:37 +0000 (0:00:00.058) 0:01:56.945 ******* 2020-11-20T13:53:37.974 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2020-11-20T13:53:37.976 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:38.006 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:38 +0000 (0:00:00.059) 0:01:57.004 ******* 2020-11-20T13:53:38.490 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2020-11-20T13:53:38.492 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"changed": false, "checksum": "a7ecef00073feab34f868065d34e9dac55134241", "dest": "/etc/chrony.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/chrony.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 215, "state": "file", "uid": 0} 2020-11-20T13:53:38.526 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:38 +0000 (0:00:00.520) 0:01:57.525 ******* 2020-11-20T13:53:38.835 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2020-11-20T13:53:38.837 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "chronyd", "state": "started", "status": {"ActiveEnterTimestamp": "Fri 2020-11-20 13:50:34 UTC", "ActiveEnterTimestampMonotonic": "19476187", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "system.slice -.mount systemd-tmpfiles-setup.service ntpdate.service systemd-journald.socket sntp.service ntpd.service sysinit.target tmp.mount basic.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Fri 2020-11-20 13:50:33 UTC", "AssertTimestampMonotonic": "18174145", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Fri 2020-11-20 13:50:33 UTC", "ConditionTimestampMonotonic": "18174113", "ConfigurationDirectoryMode": "0755", "Conflicts": "systemd-timesyncd.service shutdown.target ntpd.service", "ControlGroup": "/system.slice/chronyd.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description" 2020-11-20T13:53:38.838 INFO:teuthology.task.ansible.out:: "NTP client/server", "DevicePolicy": "auto", "Documentation": "man:chronyd(8) man:chrony.conf(5)", "DynamicUser": "no", "EnvironmentFiles": "/etc/sysconfig/chronyd (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "2439", "ExecMainStartTimestamp": "Fri 2020-11-20 13:50:34 UTC", "ExecMainStartTimestampMonotonic": "19405894", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/sbin/chronyd ; argv[]=/usr/sbin/chronyd $OPTIONS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartPost": "{ path=/usr/libexec/chrony-helper ; argv[]=/usr/libexec/chrony-helper update-daemon ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/chronyd.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "chronyd.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Fri 2020-11-20 13:50:33 UTC", "InactiveExitTimestampMonotonic": "18187884", "InvocationID": "83195762852a4f3294edc8d61e293f84", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "167772 2020-11-20T13:53:38.840 INFO:teuthology.task.ansible.out:16", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127450", "LimitNPROCSoft": "127450", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127450", "LimitSIGPENDINGSoft": "127450", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "2439", "MemoryAccounting": "yes", "MemoryCurrent": "2887680", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "chronyd.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PIDFile": "/run/chrony/chronyd.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "yes", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "full", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice -.mount sysinit.target", "RequiresMountsFor": "/var/tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardE 2020-11-20T13:53:38.842 INFO:teuthology.task.ansible.out:rror": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Fri 2020-11-20 13:50:34 UTC", "StateChangeTimestampMonotonic": "19476187", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Fri 2020-11-20 13:50:34 UTC", "WatchdogTimestampMonotonic": "19405909", "WatchdogUSec": "0"}} 2020-11-20T13:53:38.870 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:38 +0000 (0:00:00.344) 0:01:57.869 ******* 2020-11-20T13:53:38.896 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-20T13:53:38.898 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2020-11-20T13:53:38.928 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:38 +0000 (0:00:00.058) 0:01:57.927 ******* 2020-11-20T13:53:38.955 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-20T13:53:38.956 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"ansible_facts": {"ansible_python_interpreter": "/usr/bin/python3", "pip_executable": "pip3", "pip_version": "python3-pip"}, "changed": false} 2020-11-20T13:53:38.988 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:38 +0000 (0:00:00.059) 0:01:57.987 ******* 2020-11-20T13:53:39.017 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2020-11-20T13:53:39.019 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:39.050 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:39 +0000 (0:00:00.061) 0:01:58.048 ******* 2020-11-20T13:53:39.077 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2020-11-20T13:53:39.079 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:39.108 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:39 +0000 (0:00:00.058) 0:01:58.106 ******* 2020-11-20T13:53:39.134 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2020-11-20T13:53:39.135 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:39.168 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:39 +0000 (0:00:00.059) 0:01:58.166 ******* 2020-11-20T13:53:39.174 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2020-11-20T13:53:39.175 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip3", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.\n", "stderr_lines": ["WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead."], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.6/site-packages\nRequirement already satisfied: execnet in /usr/local/lib/python3.6/site-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.6/site-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.6/site-packages", "Requirement already satisfied: execnet in /usr/local/lib/python3.6/site-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.6/site-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2020-11-20T13:53:39.209 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:39 +0000 (0:00:00.041) 0:01:58.208 ******* 2020-11-20T13:53:39.235 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2020-11-20T13:53:39.236 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:39.272 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:39 +0000 (0:00:00.062) 0:01:58.270 ******* 2020-11-20T13:53:39.298 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2020-11-20T13:53:39.299 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2020-11-20T13:53:39.331 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:39 +0000 (0:00:00.058) 0:01:58.329 ******* 2020-11-20T13:53:39.367 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2020-11-20T13:53:39.369 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 4096, "state": "directory", "uid": 1000} 2020-11-20T13:53:39.401 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:39 +0000 (0:00:00.069) 0:01:58.399 ******* 2020-11-20T13:53:39.510 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2020-11-20T13:53:39.512 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2020-11-20T13:53:39.545 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:39 +0000 (0:00:00.144) 0:01:58.543 ******* 2020-11-20T13:53:39.571 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2020-11-20T13:53:39.573 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:39.604 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:39 +0000 (0:00:00.059) 0:01:58.602 ******* 2020-11-20T13:53:39.630 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2020-11-20T13:53:39.632 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-11-20T13:53:39.663 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:39 +0000 (0:00:00.058) 0:01:58.661 ******* 2020-11-20T13:53:39.688 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2020-11-20T13:53:39.690 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:39.721 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:39 +0000 (0:00:00.058) 0:01:58.720 ******* 2020-11-20T13:53:39.883 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2020-11-20T13:53:39.884 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 73, "state": "file", "uid": 1000} 2020-11-20T13:53:39.919 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:39 +0000 (0:00:00.198) 0:01:58.918 ******* 2020-11-20T13:53:39.925 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2020-11-20T13:53:39.927 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-11-20T13:53:39.961 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:39 +0000 (0:00:00.041) 0:01:58.960 ******* 2020-11-20T13:53:40.154 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2020-11-20T13:53:40.155 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2020-11-20T13:53:40.186 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:40 +0000 (0:00:00.224) 0:01:59.185 ******* 2020-11-20T13:53:40.212 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2020-11-20T13:53:40.214 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:40.245 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:40 +0000 (0:00:00.058) 0:01:59.243 ******* 2020-11-20T13:53:40.270 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2020-11-20T13:53:40.272 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2020-11-20T13:53:40.281 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} 2020-11-20T13:53:40.283 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2020-11-20T13:53:40.292 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2020-11-20T13:53:40.294 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} 2020-11-20T13:53:40.326 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:40 +0000 (0:00:00.081) 0:01:59.325 ******* 2020-11-20T13:53:40.363 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2020-11-20T13:53:40.365 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:40.375 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:40.386 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:40.387 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:40.418 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:40 +0000 (0:00:00.092) 0:01:59.417 ******* 2020-11-20T13:53:40.444 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-20T13:53:40.445 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:40.478 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:40 +0000 (0:00:00.059) 0:01:59.476 ******* 2020-11-20T13:53:40.508 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2020-11-20T13:53:40.511 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:40.541 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:40 +0000 (0:00:00.062) 0:01:59.539 ******* 2020-11-20T13:53:40.567 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2020-11-20T13:53:40.569 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:40.602 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:40 +0000 (0:00:00.061) 0:01:59.601 ******* 2020-11-20T13:53:40.629 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2020-11-20T13:53:40.631 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:40.662 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:40 +0000 (0:00:00.059) 0:01:59.661 ******* 2020-11-20T13:53:40.688 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:40.722 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:40 +0000 (0:00:00.060) 0:01:59.721 ******* 2020-11-20T13:53:40.749 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-11-20T13:53:40.751 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:40.786 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:40 +0000 (0:00:00.063) 0:01:59.784 ******* 2020-11-20T13:53:40.813 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2020-11-20T13:53:40.815 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:40.853 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:40 +0000 (0:00:00.067) 0:01:59.852 ******* 2020-11-20T13:53:40.882 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-11-20T13:53:40.884 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:40.917 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:40 +0000 (0:00:00.061) 0:01:59.914 ******* 2020-11-20T13:53:41.027 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2020-11-20T13:53:41.029 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip3", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.\n", "stderr_lines": ["WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead."], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.6/site-packages\nRequirement already satisfied: execnet in /usr/local/lib/python3.6/site-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.6/site-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.6/site-packages", "Requirement already satisfied: execnet in /usr/local/lib/python3.6/site-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.6/site-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2020-11-20T13:53:41.062 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:41 +0000 (0:00:00.146) 0:02:00.060 ******* 2020-11-20T13:53:41.089 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2020-11-20T13:53:41.091 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:41.122 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:41 +0000 (0:00:00.060) 0:02:00.120 ******* 2020-11-20T13:53:41.138 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-11-20T13:53:41.140 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.004851", "end": "2020-11-20 13:53:41.097077", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-11-20 13:53:41.092226", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T13:53:41.150 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ ok: [smithi040.front.sepia.ceph.com] => {"ansible_facts": {"interface": "ens1f0"}, "changed": false} 2020-11-20T13:53:41.183 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:41 +0000 (0:00:00.060) 0:02:00.181 ******* 2020-11-20T13:53:41.302 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-11-20T13:53:41.304 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.002965", "end": "2020-11-20 13:53:41.272478", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-11-20 13:53:41.269513", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T13:53:41.376 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2020-11-20T13:53:41.378 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2020-11-20T13:53:41.410 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:41 +0000 (0:00:00.226) 0:02:00.408 ******* 2020-11-20T13:53:41.436 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2020-11-20T13:53:41.437 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:41.467 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:41 +0000 (0:00:00.057) 0:02:00.466 ******* 2020-11-20T13:53:41.484 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-11-20T13:53:41.485 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.002782", "end": "2020-11-20 13:53:41.446433", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-11-20 13:53:41.443651", "stderr": "find: ‘/etc/ceph’: No such file or directory", "stderr_lines": ["find: ‘/etc/ceph’: No such file or directory"], "stdout": "", "stdout_lines": []} 2020-11-20T13:53:41.505 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2020-11-20T13:53:41.507 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-11-20T13:53:41.543 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:41 +0000 (0:00:00.075) 0:02:00.542 ******* 2020-11-20T13:53:41.569 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2020-11-20T13:53:41.571 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:41.602 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:41 +0000 (0:00:00.059) 0:02:00.601 ******* 2020-11-20T13:53:41.691 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-11-20T13:53:41.693 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.040340", "end": "2020-11-20 13:53:41.659317", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-11-20 13:53:41.618977", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T13:53:41.727 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:41 +0000 (0:00:00.124) 0:02:00.725 ******* 2020-11-20T13:53:41.753 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-11-20T13:53:41.786 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:41 +0000 (0:00:00.059) 0:02:00.785 ******* 2020-11-20T13:53:41.802 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2020-11-20T13:53:41.804 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-11-20T13:53:41.837 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:41 +0000 (0:00:00.050) 0:02:00.835 ******* 2020-11-20T13:53:41.968 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2020-11-20T13:53:41.970 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_runtime_t:s0", "size": 0, "state": "file", "uid": 0} 2020-11-20T13:53:42.045 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2020-11-20T13:53:42.047 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2020-11-20T13:53:42.077 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:42 +0000 (0:00:00.240) 0:02:01.076 ******* 2020-11-20T13:53:42.104 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2020-11-20T13:53:42.105 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:42.136 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:42 +0000 (0:00:00.058) 0:02:01.134 ******* 2020-11-20T13:53:42.162 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2020-11-20T13:53:42.164 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2020-11-20T13:53:42.173 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2020-11-20T13:53:42.174 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=ens1f0) => {"ansible_loop_var": "item", "changed": false, "item": "ens1f0", "skip_reason": "Conditional result was False"} 2020-11-20T13:53:42.184 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=ens1f1) => {"ansible_loop_var": "item", "changed": false, "item": "ens1f1", "skip_reason": "Conditional result was False"} 2020-11-20T13:53:42.186 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2020-11-20T13:53:42.221 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:42 +0000 (0:00:00.085) 0:02:01.220 ******* 2020-11-20T13:53:42.259 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2020-11-20T13:53:42.261 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi040.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:42.271 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'ens1f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "ens1f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:42.273 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'ens1f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "ens1f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:42.282 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:42.317 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:42 +0000 (0:00:00.096) 0:02:01.316 ******* 2020-11-20T13:53:42.343 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-20T13:53:42.345 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:42.378 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:42 +0000 (0:00:00.060) 0:02:01.377 ******* 2020-11-20T13:53:42.404 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2020-11-20T13:53:42.406 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:42.437 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:42 +0000 (0:00:00.058) 0:02:01.435 ******* 2020-11-20T13:53:42.463 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2020-11-20T13:53:42.465 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:42.498 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:42 +0000 (0:00:00.061) 0:02:01.497 ******* 2020-11-20T13:53:42.525 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:42.562 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:42 +0000 (0:00:00.063) 0:02:01.560 ******* 2020-11-20T13:53:42.588 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2020-11-20T13:53:42.590 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:42.622 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:42 +0000 (0:00:00.060) 0:02:01.620 ******* 2020-11-20T13:53:42.648 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-11-20T13:53:42.650 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:42.681 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:42 +0000 (0:00:00.059) 0:02:01.679 ******* 2020-11-20T13:53:42.707 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:42.742 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:42 +0000 (0:00:00.060) 0:02:01.740 ******* 2020-11-20T13:53:42.768 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-11-20T13:53:42.770 INFO:teuthology.task.ansible.out:skipping: [smithi040.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:42.801 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:42 +0000 (0:00:00.059) 0:02:01.799 ******* 2020-11-20T13:53:43.033 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-11-20T13:53:43.035 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.005308", "end": "2020-11-20 13:53:42.997092", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-11-20 13:53:42.991784", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T13:53:44.225 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:01.004449", "end": "2020-11-20 13:53:44.190513", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-11-20 13:53:43.186064", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T13:53:44.421 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.002952", "end": "2020-11-20 13:53:44.394117", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-11-20 13:53:44.391165", "stderr": "find: ‘/etc/ceph’: No such file or directory", "stderr_lines": ["find: ‘/etc/ceph’: No such file or directory"], "stdout": "", "stdout_lines": []} 2020-11-20T13:53:45.640 INFO:teuthology.task.ansible.out:ok: [smithi040.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:01.027082", "end": "2020-11-20 13:53:45.603934", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-11-20 13:53:44.576852", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T13:53:45.674 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:45 +0000 (0:00:02.873) 0:02:04.673 ******* 2020-11-20T13:53:45.701 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-11-20T13:53:45.733 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:45 +0000 (0:00:00.058) 0:02:04.732 ******* 2020-11-20T13:53:45.946 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2020-11-20T13:53:45.947 INFO:teuthology.task.ansible.out:changed: [smithi040.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_runtime_t:s0", "size": 0, "state": "file", "uid": 0} 2020-11-20T13:53:46.036 INFO:teuthology.task.ansible.out: PLAY [cobbler] ***************************************************************** skipping: no hosts matched 2020-11-20T13:53:46.040 INFO:teuthology.task.ansible.out: PLAY [paddles] ***************************************************************** skipping: no hosts matched 2020-11-20T13:53:46.044 INFO:teuthology.task.ansible.out: PLAY [pulpito] ***************************************************************** skipping: no hosts matched 2020-11-20T13:53:46.045 INFO:teuthology.task.ansible.out: PLAY RECAP ********************************************************************* smithi040.front.sepia.ceph.com : ok=115 changed=33 unreachable=0 failed=0 skipped=181 rescued=0 ignored=3 smithi116.front.sepia.ceph.com : ok=115 changed=33 unreachable=0 failed=0 skipped=181 rescued=0 ignored=3 Friday 20 November 2020 13:53:46 +0000 (0:00:00.310) 0:02:05.043 ******* =============================================================================== common : nrpe - Remove SELinux policy package -------------------------- 15.88s common : nrpe - Load SELinux policy package ---------------------------- 14.88s testnode : Zap all non-root disks --------------------------------------- 5.33s common : Make sure smartmontools is installed --------------------------- 3.22s testnode : List any leftover Ceph artifacts from previous jobs ---------- 2.87s Gathering Facts --------------------------------------------------------- 2.47s testnode : Ensure ceph dependency packages are not present. ------------- 2.01s testnode : Ensure ceph packages are not present. ------------------------ 1.83s common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/. --- 1.43s testnode : Configure version specific repos in /etc/yum.repos.d/ -------- 1.42s common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/. --- 1.24s common : Install Kerberos Packages (RedHat) ----------------------------- 1.17s testnode : Make sure rpm dependencies are installed --------------------- 1.03s testnode : Rebuild rpmdb ------------------------------------------------ 1.03s testnode : Configure local mirrorlists ---------------------------------- 1.01s common : Configure epel repos in /etc/yum.repos.d/ ---------------------- 0.86s common : Configure local epel mirrorlists ------------------------------- 0.81s users : Create all admin users with sudo access. ------------------------ 0.78s ansible-managed 2020-11-20T13:53:46.047 INFO:teuthology.task.ansible.out: : Create the cephlab_sudo sudoers.d file. --------------- 0.78s Gathering Facts --------------------------------------------------------- 0.71s 2020-11-20T13:53:46.335 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi040.front.sepia.ceph.com'), Remote(name='ubuntu@smithi116.front.sepia.ceph.com')] 2020-11-20T13:53:46.336 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-20T13:53:46.337 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T13:53:46.407 INFO:teuthology.orchestra.run.smithi040:> true 2020-11-20T13:53:46.494 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-20T13:53:46.495 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T13:53:46.565 INFO:teuthology.orchestra.run.smithi116:> true 2020-11-20T13:53:46.654 INFO:teuthology.run_tasks:Running task clock... 2020-11-20T13:53:46.665 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2020-11-20T13:53:46.666 INFO:teuthology.orchestra.run:Running command with timeout 360 2020-11-20T13:53:46.666 INFO:teuthology.orchestra.run.smithi040:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-11-20T13:53:46.708 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2020-11-20T13:53:46.730 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2020-11-20T13:53:46.778 INFO:teuthology.orchestra.run.smithi040.stderr:sudo: ntpd: command not found 2020-11-20T13:53:46.796 INFO:teuthology.orchestra.run.smithi040.stdout:506 Cannot talk to daemon 2020-11-20T13:53:46.818 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to start ntp.service: Unit ntp.service not found. 2020-11-20T13:53:46.838 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2020-11-20T13:53:46.900 INFO:teuthology.orchestra.run.smithi040.stderr:bash: ntpq: command not found 2020-11-20T13:53:46.902 INFO:teuthology.orchestra.run.smithi040.stdout:210 Number of sources = 4 2020-11-20T13:53:46.902 INFO:teuthology.orchestra.run.smithi040.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2020-11-20T13:53:46.903 INFO:teuthology.orchestra.run.smithi040.stdout:=============================================================================== 2020-11-20T13:53:46.903 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-11-20T13:53:46.903 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-11-20T13:53:46.903 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-11-20T13:53:46.903 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-11-20T13:53:46.904 INFO:teuthology.orchestra.run:Running command with timeout 360 2020-11-20T13:53:46.905 INFO:teuthology.orchestra.run.smithi116:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-11-20T13:53:46.945 INFO:teuthology.orchestra.run.smithi116.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2020-11-20T13:53:46.964 INFO:teuthology.orchestra.run.smithi116.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2020-11-20T13:53:47.004 INFO:teuthology.orchestra.run.smithi116.stderr:sudo: ntpd: command not found 2020-11-20T13:53:47.018 INFO:teuthology.orchestra.run.smithi116.stdout:506 Cannot talk to daemon 2020-11-20T13:53:47.037 INFO:teuthology.orchestra.run.smithi116.stderr:Failed to start ntp.service: Unit ntp.service not found. 2020-11-20T13:53:47.054 INFO:teuthology.orchestra.run.smithi116.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2020-11-20T13:53:47.117 INFO:teuthology.orchestra.run.smithi116.stderr:bash: ntpq: command not found 2020-11-20T13:53:47.120 INFO:teuthology.orchestra.run.smithi116.stdout:210 Number of sources = 4 2020-11-20T13:53:47.120 INFO:teuthology.orchestra.run.smithi116.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2020-11-20T13:53:47.120 INFO:teuthology.orchestra.run.smithi116.stdout:=============================================================================== 2020-11-20T13:53:47.121 INFO:teuthology.orchestra.run.smithi116.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-11-20T13:53:47.121 INFO:teuthology.orchestra.run.smithi116.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-11-20T13:53:47.121 INFO:teuthology.orchestra.run.smithi116.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-11-20T13:53:47.121 INFO:teuthology.orchestra.run.smithi116.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-11-20T13:53:47.122 INFO:teuthology.run_tasks:Running task install... 2020-11-20T13:53:47.131 DEBUG:teuthology.task.install:project ceph 2020-11-20T13:53:47.131 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'sha1': 'a7b62340ad27b26642bb54ec916a97c7e1f07a34'}} 2020-11-20T13:53:47.132 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'sha1': 'a7b62340ad27b26642bb54ec916a97c7e1f07a34'} 2020-11-20T13:53:47.132 INFO:teuthology.task.install:Using flavor: basic 2020-11-20T13:53:47.140 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2020-11-20T13:53:47.141 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']} 2020-11-20T13:53:47.141 INFO:teuthology.orchestra.run.smithi040:> sudo lsb_release -is 2020-11-20T13:53:47.184 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS 2020-11-20T13:53:47.185 DEBUG:teuthology.misc:System to be installed: CentOS 2020-11-20T13:53:47.186 INFO:teuthology.orchestra.run.smithi116:> sudo lsb_release -is 2020-11-20T13:53:47.186 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'flavor': 'basic', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a7b62340ad27b26642bb54ec916a97c7e1f07a34', 'tag': None, 'wait_for_package': False} 2020-11-20T13:53:47.187 INFO:teuthology.orchestra.run.smithi040:> sudo lsb_release -is 2020-11-20T13:53:47.223 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS 2020-11-20T13:53:47.224 DEBUG:teuthology.misc:System to be installed: CentOS 2020-11-20T13:53:47.225 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'flavor': 'basic', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a7b62340ad27b26642bb54ec916a97c7e1f07a34', 'tag': None, 'wait_for_package': False} 2020-11-20T13:53:47.225 INFO:teuthology.orchestra.run.smithi116:> sudo lsb_release -is 2020-11-20T13:53:47.267 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS 2020-11-20T13:53:47.268 DEBUG:teuthology.misc:System to be installed: CentOS 2020-11-20T13:53:47.268 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a7b62340ad27b26642bb54ec916a97c7e1f07a34 2020-11-20T13:53:47.304 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS 2020-11-20T13:53:47.304 DEBUG:teuthology.misc:System to be installed: CentOS 2020-11-20T13:53:47.305 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a7b62340ad27b26642bb54ec916a97c7e1f07a34 2020-11-20T13:53:47.437 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/octopus/a7b62340ad27b26642bb54ec916a97c7e1f07a34/centos/8/flavors/default/ 2020-11-20T13:53:47.437 INFO:teuthology.task.install.rpm:Package version is 15.2.5-588.ga7b62340 2020-11-20T13:53:47.462 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/octopus/a7b62340ad27b26642bb54ec916a97c7e1f07a34/centos/8/flavors/default/ 2020-11-20T13:53:47.463 INFO:teuthology.task.install.rpm:Package version is 15.2.5-588.ga7b62340 2020-11-20T13:53:47.566 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/octopus/a7b62340ad27b26642bb54ec916a97c7e1f07a34/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/octopus/a7b62340ad27b26642bb54ec916a97c7e1f07a34/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/octopus/a7b62340ad27b26642bb54ec916a97c7e1f07a34/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2020-11-20T13:53:47.567 INFO:teuthology.orchestra.run.smithi040:> set -ex 2020-11-20T13:53:47.567 INFO:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/yum.repos.d/ceph.repo 2020-11-20T13:53:47.582 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/octopus/a7b62340ad27b26642bb54ec916a97c7e1f07a34/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/octopus/a7b62340ad27b26642bb54ec916a97c7e1f07a34/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/octopus/a7b62340ad27b26642bb54ec916a97c7e1f07a34/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2020-11-20T13:53:47.582 INFO:teuthology.orchestra.run.smithi116:> set -ex 2020-11-20T13:53:47.583 INFO:teuthology.orchestra.run.smithi116:> sudo dd of=/etc/yum.repos.d/ceph.repo 2020-11-20T13:53:47.584 INFO:teuthology.orchestra.run.smithi040.stderr:+ sudo dd of=/etc/yum.repos.d/ceph.repo 2020-11-20T13:53:47.596 INFO:teuthology.orchestra.run.smithi116.stderr:+ sudo dd of=/etc/yum.repos.d/ceph.repo 2020-11-20T13:53:47.599 INFO:teuthology.orchestra.run.smithi040.stderr:1+1 records in 2020-11-20T13:53:47.599 INFO:teuthology.orchestra.run.smithi040.stderr:1+1 records out 2020-11-20T13:53:47.599 INFO:teuthology.orchestra.run.smithi040.stderr:589 bytes copied, 4.5067e-05 s, 13.1 MB/s 2020-11-20T13:53:47.600 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, bison, flex, elfutils-libelf-devel, openssl-devel, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2020-11-20T13:53:47.601 INFO:teuthology.orchestra.run.smithi040:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a7b62340ad27b26642bb54ec916a97c7e1f07a34/;g' /etc/yum.repos.d/ceph.repo ; fi 2020-11-20T13:53:47.610 INFO:teuthology.orchestra.run.smithi116.stderr:1+1 records in 2020-11-20T13:53:47.610 INFO:teuthology.orchestra.run.smithi116.stderr:1+1 records out 2020-11-20T13:53:47.610 INFO:teuthology.orchestra.run.smithi116.stderr:589 bytes copied, 4.0541e-05 s, 14.5 MB/s 2020-11-20T13:53:47.611 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, bison, flex, elfutils-libelf-devel, openssl-devel, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2020-11-20T13:53:47.611 INFO:teuthology.orchestra.run.smithi116:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a7b62340ad27b26642bb54ec916a97c7e1f07a34/;g' /etc/yum.repos.d/ceph.repo ; fi 2020-11-20T13:53:47.673 INFO:teuthology.orchestra.run.smithi040:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2020-11-20T13:53:47.679 INFO:teuthology.orchestra.run.smithi116:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2020-11-20T13:53:47.760 INFO:teuthology.orchestra.run.smithi116:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2020-11-20T13:53:47.762 INFO:teuthology.orchestra.run.smithi040:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2020-11-20T13:53:47.792 INFO:teuthology.orchestra.run.smithi040.stdout:check_obsoletes = 1 2020-11-20T13:53:47.793 INFO:teuthology.orchestra.run.smithi040:> sudo yum clean all 2020-11-20T13:53:47.826 INFO:teuthology.orchestra.run.smithi116.stdout:check_obsoletes = 1 2020-11-20T13:53:47.827 INFO:teuthology.orchestra.run.smithi116:> sudo yum clean all 2020-11-20T13:53:48.089 INFO:teuthology.orchestra.run.smithi040.stdout:42 files removed 2020-11-20T13:53:48.106 INFO:teuthology.orchestra.run.smithi116.stdout:42 files removed 2020-11-20T13:53:48.119 INFO:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs bison flex elfutils-libelf-devel openssl-devel libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2020-11-20T13:53:48.135 INFO:teuthology.orchestra.run.smithi116:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs bison flex elfutils-libelf-devel openssl-devel libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2020-11-20T13:53:48.724 INFO:teuthology.orchestra.run.smithi116.stdout:ceph packages for x86_64 279 kB/s | 69 kB 00:00 2020-11-20T13:53:48.738 INFO:teuthology.orchestra.run.smithi040.stdout:ceph packages for x86_64 277 kB/s | 69 kB 00:00 2020-11-20T13:53:48.962 INFO:teuthology.orchestra.run.smithi116.stdout:ceph noarch packages 70 kB/s | 15 kB 00:00 2020-11-20T13:53:48.977 INFO:teuthology.orchestra.run.smithi040.stdout:ceph noarch packages 70 kB/s | 15 kB 00:00 2020-11-20T13:53:49.189 INFO:teuthology.orchestra.run.smithi116.stdout:ceph source packages 6.9 kB/s | 1.4 kB 00:00 2020-11-20T13:53:49.207 INFO:teuthology.orchestra.run.smithi040.stdout:ceph source packages 6.9 kB/s | 1.4 kB 00:00 2020-11-20T13:53:49.384 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS-8 - AppStream 40 MB/s | 7.0 MB 00:00 2020-11-20T13:53:49.429 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS-8 - AppStream 35 MB/s | 7.0 MB 00:00 2020-11-20T13:53:50.578 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS-8 - Base 26 MB/s | 2.2 MB 00:00 2020-11-20T13:53:50.683 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS-8 - Base 25 MB/s | 2.2 MB 00:00 2020-11-20T13:53:51.004 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS-8 - Extras 87 kB/s | 5.5 kB 00:00 2020-11-20T13:53:51.133 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS-8 - Extras 86 kB/s | 5.5 kB 00:00 2020-11-20T13:53:51.464 INFO:teuthology.orchestra.run.smithi116.stdout:Extra Packages for Enterprise Linux 20 MB/s | 8.3 MB 00:00 2020-11-20T13:53:51.542 INFO:teuthology.orchestra.run.smithi040.stdout:Extra Packages for Enterprise Linux 23 MB/s | 8.3 MB 00:00 2020-11-20T13:53:53.189 INFO:teuthology.orchestra.run.smithi116.stdout:lab-extras 4.6 MB/s | 23 kB 00:00 2020-11-20T13:53:53.362 INFO:teuthology.orchestra.run.smithi040.stdout:lab-extras 4.8 MB/s | 23 kB 00:00 2020-11-20T13:53:54.071 INFO:teuthology.orchestra.run.smithi116.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2020-11-20T13:53:54.072 INFO:teuthology.orchestra.run.smithi116.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2020-11-20T13:53:54.075 INFO:teuthology.orchestra.run.smithi116.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2020-11-20T13:53:54.076 INFO:teuthology.orchestra.run.smithi116.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2020-11-20T13:53:54.076 INFO:teuthology.orchestra.run.smithi116.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2020-11-20T13:53:54.077 INFO:teuthology.orchestra.run.smithi116.stdout:Package xfsprogs-5.0.0-1.el8.x86_64 is already installed. 2020-11-20T13:53:54.205 INFO:teuthology.orchestra.run.smithi116.stdout:Dependencies resolved. 2020-11-20T13:53:54.224 INFO:teuthology.orchestra.run.smithi116.stdout:================================================================================ 2020-11-20T13:53:54.225 INFO:teuthology.orchestra.run.smithi116.stdout: Package ArchVersion Repository Size 2020-11-20T13:53:54.225 INFO:teuthology.orchestra.run.smithi116.stdout:================================================================================ 2020-11-20T13:53:54.225 INFO:teuthology.orchestra.run.smithi116.stdout:Installing: 2020-11-20T13:53:54.226 INFO:teuthology.orchestra.run.smithi116.stdout: ceph x86_642:15.2.5-588.ga7b62340.el8 ceph 6.4 k 2020-11-20T13:53:54.226 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-base x86_642:15.2.5-588.ga7b62340.el8 ceph 5.6 M 2020-11-20T13:53:54.226 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-fuse x86_642:15.2.5-588.ga7b62340.el8 ceph 577 k 2020-11-20T13:53:54.226 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-immutable-object-cache x86_642:15.2.5-588.ga7b62340.el8 ceph 205 k 2020-11-20T13:53:54.226 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr x86_642:15.2.5-588.ga7b62340.el8 ceph 1.4 M 2020-11-20T13:53:54.227 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-radosgw x86_642:15.2.5-588.ga7b62340.el8 ceph 10 M 2020-11-20T13:53:54.227 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-test x86_642:15.2.5-588.ga7b62340.el8 ceph 42 M 2020-11-20T13:53:54.227 INFO:teuthology.orchestra.run.smithi116.stdout: cephadm x86_642:15.2.5-588.ga7b62340.el8 ceph 57 k 2020-11-20T13:53:54.227 INFO:teuthology.orchestra.run.smithi116.stdout: libcephfs-devel x86_642:15.2.5-588.ga7b62340.el8 ceph 22 k 2020-11-20T13:53:54.227 INFO:teuthology.orchestra.run.smithi116.stdout: libcephfs2 x86_642:15.2.5-588.ga7b62340.el8 ceph 511 k 2020-11-20T13:53:54.228 INFO:teuthology.orchestra.run.smithi116.stdout: librados-devel x86_642:15.2.5-588.ga7b62340.el8 ceph 141 k 2020-11-20T13:53:54.228 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cephfs x86_642:15.2.5-588.ga7b62340.el8 ceph 163 k 2020-11-20T13:53:54.228 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rados x86_642:15.2.5-588.ga7b62340.el8 ceph 351 k 2020-11-20T13:53:54.228 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rbd x86_642:15.2.5-588.ga7b62340.el8 ceph 334 k 2020-11-20T13:53:54.228 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rgw x86_642:15.2.5-588.ga7b62340.el8 ceph 113 k 2020-11-20T13:53:54.229 INFO:teuthology.orchestra.run.smithi116.stdout: rbd-fuse x86_642:15.2.5-588.ga7b62340.el8 ceph 72 k 2020-11-20T13:53:54.229 INFO:teuthology.orchestra.run.smithi116.stdout: rbd-mirror x86_642:15.2.5-588.ga7b62340.el8 ceph 2.7 M 2020-11-20T13:53:54.229 INFO:teuthology.orchestra.run.smithi116.stdout: rbd-nbd x86_642:15.2.5-588.ga7b62340.el8 ceph 141 k 2020-11-20T13:53:54.229 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-cephadm noarch2:15.2.5-588.ga7b62340.el8 ceph-noarch 65 k 2020-11-20T13:53:54.230 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-dashboard noarch2:15.2.5-588.ga7b62340.el8 ceph-noarch 3.5 M 2020-11-20T13:53:54.230 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-diskprediction-local noarch2:15.2.5-588.ga7b62340.el8 ceph-noarch 8.6 M 2020-11-20T13:53:54.230 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-rook noarch2:15.2.5-588.ga7b62340.el8 ceph-noarch 27 k 2020-11-20T13:53:54.230 INFO:teuthology.orchestra.run.smithi116.stdout: bison x86_643.0.4-10.el8 CentOS-AppStream 688 k 2020-11-20T13:53:54.231 INFO:teuthology.orchestra.run.smithi116.stdout: flex x86_642.6.1-9.el8 CentOS-AppStream 320 k 2020-11-20T13:53:54.231 INFO:teuthology.orchestra.run.smithi116.stdout: libtool x86_642.4.6-25.el8 CentOS-AppStream 709 k 2020-11-20T13:53:54.231 INFO:teuthology.orchestra.run.smithi116.stdout: elfutils-libelf-devel x86_640.176-5.el8 CentOS-Base 54 k 2020-11-20T13:53:54.231 INFO:teuthology.orchestra.run.smithi116.stdout: libaio-devel x86_640.3.112-1.el8 CentOS-Base 19 k 2020-11-20T13:53:54.231 INFO:teuthology.orchestra.run.smithi116.stdout: libuuid-devel x86_642.32.1-17.el8 CentOS-Base 95 k 2020-11-20T13:53:54.232 INFO:teuthology.orchestra.run.smithi116.stdout: openssl-devel x86_641:1.1.1c-2.el8_1.1 CentOS-Base 2.3 M 2020-11-20T13:53:54.232 INFO:teuthology.orchestra.run.smithi116.stdout: xfsprogs-devel x86_645.0.0-1.el8 CentOS-Base 69 k 2020-11-20T13:53:54.232 INFO:teuthology.orchestra.run.smithi116.stdout:Upgrading: 2020-11-20T13:53:54.232 INFO:teuthology.orchestra.run.smithi116.stdout: librados2 x86_642:15.2.5-588.ga7b62340.el8 ceph 3.6 M 2020-11-20T13:53:54.232 INFO:teuthology.orchestra.run.smithi116.stdout: librbd1 x86_642:15.2.5-588.ga7b62340.el8 ceph 1.9 M 2020-11-20T13:53:54.233 INFO:teuthology.orchestra.run.smithi116.stdout:Installing dependencies: 2020-11-20T13:53:54.233 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-common x86_642:15.2.5-588.ga7b62340.el8 ceph 21 M 2020-11-20T13:53:54.233 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mds x86_642:15.2.5-588.ga7b62340.el8 ceph 2.0 M 2020-11-20T13:53:54.233 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mon x86_642:15.2.5-588.ga7b62340.el8 ceph 4.4 M 2020-11-20T13:53:54.234 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-osd x86_642:15.2.5-588.ga7b62340.el8 ceph 17 M 2020-11-20T13:53:54.234 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-selinux x86_642:15.2.5-588.ga7b62340.el8 ceph 25 k 2020-11-20T13:53:54.234 INFO:teuthology.orchestra.run.smithi116.stdout: libradosstriper1 x86_642:15.2.5-588.ga7b62340.el8 ceph 381 k 2020-11-20T13:53:54.234 INFO:teuthology.orchestra.run.smithi116.stdout: librgw2 x86_642:15.2.5-588.ga7b62340.el8 ceph 3.2 M 2020-11-20T13:53:54.234 INFO:teuthology.orchestra.run.smithi116.stdout: python3-ceph-argparse x86_642:15.2.5-588.ga7b62340.el8 ceph 40 k 2020-11-20T13:53:54.235 INFO:teuthology.orchestra.run.smithi116.stdout: python3-ceph-common x86_642:15.2.5-588.ga7b62340.el8 ceph 73 k 2020-11-20T13:53:54.235 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-grafana-dashboards noarch2:15.2.5-588.ga7b62340.el8 ceph-noarch 24 k 2020-11-20T13:53:54.235 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-modules-core noarch2:15.2.5-588.ga7b62340.el8 ceph-noarch 176 k 2020-11-20T13:53:54.235 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-prometheus-alerts noarch2:15.2.5-588.ga7b62340.el8 ceph-noarch 8.7 k 2020-11-20T13:53:54.235 INFO:teuthology.orchestra.run.smithi116.stdout: automake noarch1.16.1-6.el8 CentOS-AppStream 713 k 2020-11-20T13:53:54.236 INFO:teuthology.orchestra.run.smithi116.stdout: jq x86_641.5-12.el8 CentOS-AppStream 161 k 2020-11-20T13:53:54.236 INFO:teuthology.orchestra.run.smithi116.stdout: librdkafka x86_640.11.4-1.el8 CentOS-AppStream 353 k 2020-11-20T13:53:54.236 INFO:teuthology.orchestra.run.smithi116.stdout: lttng-ust x86_642.8.1-11.el8 CentOS-AppStream 259 k 2020-11-20T13:53:54.236 INFO:teuthology.orchestra.run.smithi116.stdout: oniguruma x86_646.8.2-1.el8 CentOS-AppStream 188 k 2020-11-20T13:53:54.237 INFO:teuthology.orchestra.run.smithi116.stdout: openblas x86_640.3.3-2.el8 CentOS-AppStream 4.3 M 2020-11-20T13:53:54.237 INFO:teuthology.orchestra.run.smithi116.stdout: openblas-threads x86_640.3.3-2.el8 CentOS-AppStream 4.4 M 2020-11-20T13:53:54.237 INFO:teuthology.orchestra.run.smithi116.stdout: python3-babel noarch2.5.1-5.el8 CentOS-AppStream 4.8 M 2020-11-20T13:53:54.237 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jinja2 noarch2.10.1-2.el8_0 CentOS-AppStream 538 k 2020-11-20T13:53:54.237 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jsonpatch noarch1.21-2.el8 CentOS-AppStream 27 k 2020-11-20T13:53:54.238 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jsonpointer noarch1.10-11.el8 CentOS-AppStream 20 k 2020-11-20T13:53:54.238 INFO:teuthology.orchestra.run.smithi116.stdout: python3-lxml x86_644.2.3-1.el8 CentOS-AppStream 1.5 M 2020-11-20T13:53:54.238 INFO:teuthology.orchestra.run.smithi116.stdout: python3-mako noarch1.0.6-13.el8 CentOS-AppStream 157 k 2020-11-20T13:53:54.238 INFO:teuthology.orchestra.run.smithi116.stdout: python3-markupsafe x86_640.23-19.el8 CentOS-AppStream 39 k 2020-11-20T13:53:54.238 INFO:teuthology.orchestra.run.smithi116.stdout: python3-numpy x86_641:1.14.3-9.el8 CentOS-AppStream 3.7 M 2020-11-20T13:53:54.239 INFO:teuthology.orchestra.run.smithi116.stdout: python3-numpy-f2py x86_641:1.14.3-9.el8 CentOS-AppStream 225 k 2020-11-20T13:53:54.239 INFO:teuthology.orchestra.run.smithi116.stdout: python3-prettytable noarch0.7.2-14.el8 CentOS-AppStream 44 k 2020-11-20T13:53:54.239 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pyasn1 noarch0.3.7-6.el8 CentOS-AppStream 126 k 2020-11-20T13:53:54.239 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pyasn1-modules noarch0.3.7-6.el8 CentOS-AppStream 110 k 2020-11-20T13:53:54.239 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pytz noarch2017.2-9.el8 CentOS-AppStream 54 k 2020-11-20T13:53:54.240 INFO:teuthology.orchestra.run.smithi116.stdout: python3-scipy x86_641.0.0-20.module_el8.1.0+245+c39af44f CentOS-AppStream 15 M 2020-11-20T13:53:54.240 INFO:teuthology.orchestra.run.smithi116.stdout: python3-webencodings noarch0.5.1-6.el8 CentOS-AppStream 27 k 2020-11-20T13:53:54.240 INFO:teuthology.orchestra.run.smithi116.stdout: python3-werkzeug noarch0.12.2-4.el8 CentOS-AppStream 457 k 2020-11-20T13:53:54.240 INFO:teuthology.orchestra.run.smithi116.stdout: socat x86_641.7.3.2-6.el8 CentOS-AppStream 298 k 2020-11-20T13:53:54.240 INFO:teuthology.orchestra.run.smithi116.stdout: fuse x86_642.9.7-12.el8 CentOS-Base 82 k 2020-11-20T13:53:54.241 INFO:teuthology.orchestra.run.smithi116.stdout: fuse-common x86_643.2.1-12.el8 CentOS-Base 21 k 2020-11-20T13:53:54.241 INFO:teuthology.orchestra.run.smithi116.stdout: keyutils-libs-devel x86_641.5.10-6.el8 CentOS-Base 48 k 2020-11-20T13:53:54.241 INFO:teuthology.orchestra.run.smithi116.stdout: krb5-devel x86_641.17-9.el8 CentOS-Base 548 k 2020-11-20T13:53:54.241 INFO:teuthology.orchestra.run.smithi116.stdout: libcom_err-devel x86_641.44.6-3.el8 CentOS-Base 38 k 2020-11-20T13:53:54.242 INFO:teuthology.orchestra.run.smithi116.stdout: libgfortran x86_648.3.1-4.5.el8 CentOS-Base 638 k 2020-11-20T13:53:54.242 INFO:teuthology.orchestra.run.smithi116.stdout: libquadmath x86_648.3.1-4.5.el8 CentOS-Base 169 k 2020-11-20T13:53:54.242 INFO:teuthology.orchestra.run.smithi116.stdout: librabbitmq x86_640.9.0-1.el8 CentOS-Base 47 k 2020-11-20T13:53:54.242 INFO:teuthology.orchestra.run.smithi116.stdout: libselinux-devel x86_642.9-2.1.el8 CentOS-Base 199 k 2020-11-20T13:53:54.243 INFO:teuthology.orchestra.run.smithi116.stdout: libsepol-devel x86_642.9-1.el8 CentOS-Base 86 k 2020-11-20T13:53:54.243 INFO:teuthology.orchestra.run.smithi116.stdout: libverto-devel x86_640.3.0-5.el8 CentOS-Base 18 k 2020-11-20T13:53:54.243 INFO:teuthology.orchestra.run.smithi116.stdout: libxslt x86_641.1.32-3.el8 CentOS-Base 249 k 2020-11-20T13:53:54.243 INFO:teuthology.orchestra.run.smithi116.stdout: pcre2-devel x86_6410.32-1.el8 CentOS-Base 605 k 2020-11-20T13:53:54.243 INFO:teuthology.orchestra.run.smithi116.stdout: pcre2-utf16 x86_6410.32-1.el8 CentOS-Base 228 k 2020-11-20T13:53:54.244 INFO:teuthology.orchestra.run.smithi116.stdout: pcre2-utf32 x86_6410.32-1.el8 CentOS-Base 220 k 2020-11-20T13:53:54.244 INFO:teuthology.orchestra.run.smithi116.stdout: python3-chardet noarch3.0.4-7.el8 CentOS-Base 195 k 2020-11-20T13:53:54.244 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jwt noarch1.6.1-2.el8 CentOS-Base 43 k 2020-11-20T13:53:54.244 INFO:teuthology.orchestra.run.smithi116.stdout: python3-oauthlib noarch2.1.0-1.el8 CentOS-Base 155 k 2020-11-20T13:53:54.245 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pysocks noarch1.6.8-3.el8 CentOS-Base 34 k 2020-11-20T13:53:54.245 INFO:teuthology.orchestra.run.smithi116.stdout: python3-requests noarch2.20.0-2.1.el8_1 CentOS-Base 123 k 2020-11-20T13:53:54.245 INFO:teuthology.orchestra.run.smithi116.stdout: python3-requests-oauthlib noarch1.0.0-1.el8 CentOS-Base 43 k 2020-11-20T13:53:54.245 INFO:teuthology.orchestra.run.smithi116.stdout: python3-urllib3 noarch1.24.2-2.el8 CentOS-Base 176 k 2020-11-20T13:53:54.245 INFO:teuthology.orchestra.run.smithi116.stdout: zlib-devel x86_641.2.11-10.el8 CentOS-Base 56 k 2020-11-20T13:53:54.246 INFO:teuthology.orchestra.run.smithi116.stdout: fmt x86_646.2.1-1.el8 epel 116 k 2020-11-20T13:53:54.246 INFO:teuthology.orchestra.run.smithi116.stdout: gperftools-libs x86_642.7-6.el8 epel 315 k 2020-11-20T13:53:54.246 INFO:teuthology.orchestra.run.smithi116.stdout: leveldb x86_641.22-1.el8 epel 181 k 2020-11-20T13:53:54.246 INFO:teuthology.orchestra.run.smithi116.stdout: liboath x86_642.6.2-3.el8 epel 59 k 2020-11-20T13:53:54.247 INFO:teuthology.orchestra.run.smithi116.stdout: libunwind x86_641.3.1-3.el8 epel 75 k 2020-11-20T13:53:54.247 INFO:teuthology.orchestra.run.smithi116.stdout: python3-apipkg noarch1.5-6.el8 epel 21 k 2020-11-20T13:53:54.247 INFO:teuthology.orchestra.run.smithi116.stdout: python3-bcrypt x86_643.1.6-2.el8.1 epel 44 k 2020-11-20T13:53:54.247 INFO:teuthology.orchestra.run.smithi116.stdout: python3-beautifulsoup4 noarch4.6.3-2.el8.1 epel 185 k 2020-11-20T13:53:54.247 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cachetools noarch3.1.1-4.el8 epel 33 k 2020-11-20T13:53:54.248 INFO:teuthology.orchestra.run.smithi116.stdout: python3-certifi noarch2018.10.15-7.el8 epel 16 k 2020-11-20T13:53:54.248 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cheroot noarch8.2.1-1.el8 epel 145 k 2020-11-20T13:53:54.248 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cherrypy noarch18.4.0-1.el8 epel 384 k 2020-11-20T13:53:54.248 INFO:teuthology.orchestra.run.smithi116.stdout: python3-execnet noarch1.7.1-1.el8 epel 355 k 2020-11-20T13:53:54.248 INFO:teuthology.orchestra.run.smithi116.stdout: python3-google-auth noarch1:1.1.1-10.el8 epel 81 k 2020-11-20T13:53:54.249 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco noarch6.2-6.el8 epel 11 k 2020-11-20T13:53:54.249 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco-functools noarch2.0-4.el8 epel 18 k 2020-11-20T13:53:54.249 INFO:teuthology.orchestra.run.smithi116.stdout: python3-kubernetes noarch11.0.0-2.el8 epel 1.2 M 2020-11-20T13:53:54.249 INFO:teuthology.orchestra.run.smithi116.stdout: python3-logutils noarch0.3.5-11.el8 epel 49 k 2020-11-20T13:53:54.249 INFO:teuthology.orchestra.run.smithi116.stdout: python3-more-itertools noarch7.2.0-3.el8 epel 59 k 2020-11-20T13:53:54.250 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pecan noarch1.3.2-9.el8 epel 283 k 2020-11-20T13:53:54.250 INFO:teuthology.orchestra.run.smithi116.stdout: python3-portend noarch2.6-1.el8 epel 16 k 2020-11-20T13:53:54.250 INFO:teuthology.orchestra.run.smithi116.stdout: python3-remoto noarch1.1.4-4.el8 epel 48 k 2020-11-20T13:53:54.250 INFO:teuthology.orchestra.run.smithi116.stdout: python3-repoze-lru noarch0.7-6.el8 epel 34 k 2020-11-20T13:53:54.251 INFO:teuthology.orchestra.run.smithi116.stdout: python3-routes noarch2.4.1-12.el8 epel 196 k 2020-11-20T13:53:54.251 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rsa noarch3.4.2-15.el8 epel 73 k 2020-11-20T13:53:54.251 INFO:teuthology.orchestra.run.smithi116.stdout: python3-simplegeneric noarch0.8.1-17.el8 epel 19 k 2020-11-20T13:53:54.251 INFO:teuthology.orchestra.run.smithi116.stdout: python3-singledispatch noarch3.4.0.3-18.el8 epel 24 k 2020-11-20T13:53:54.252 INFO:teuthology.orchestra.run.smithi116.stdout: python3-tempora noarch1.14.1-5.el8 epel 29 k 2020-11-20T13:53:54.252 INFO:teuthology.orchestra.run.smithi116.stdout: python3-trustme noarch0.6.0-4.el8 epel 27 k 2020-11-20T13:53:54.252 INFO:teuthology.orchestra.run.smithi116.stdout: python3-waitress noarch1.2.1-2.el8.1 epel 241 k 2020-11-20T13:53:54.252 INFO:teuthology.orchestra.run.smithi116.stdout: python3-webob noarch1.8.5-1.el8.1 epel 251 k 2020-11-20T13:53:54.252 INFO:teuthology.orchestra.run.smithi116.stdout: python3-websocket-client noarch0.56.0-5.el8 epel 61 k 2020-11-20T13:53:54.253 INFO:teuthology.orchestra.run.smithi116.stdout: python3-webtest noarch2.0.33-1.el8 epel 85 k 2020-11-20T13:53:54.253 INFO:teuthology.orchestra.run.smithi116.stdout: python3-zc-lockfile noarch2.0-2.el8 epel 23 k 2020-11-20T13:53:54.253 INFO:teuthology.orchestra.run.smithi116.stdout: xmlstarlet x86_641.6.1-11.el8 epel 69 k 2020-11-20T13:53:54.253 INFO:teuthology.orchestra.run.smithi116.stdout:Installing weak dependencies: 2020-11-20T13:53:54.254 INFO:teuthology.orchestra.run.smithi116.stdout: python3-html5lib noarch1:0.999999999-6.el8 CentOS-AppStream 214 k 2020-11-20T13:53:54.254 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cssselect noarch0.9.2-10.el8 epel 40 k 2020-11-20T13:53:54.254 INFO:teuthology.orchestra.run.smithi116.stdout:Downgrading: 2020-11-20T13:53:54.254 INFO:teuthology.orchestra.run.smithi116.stdout: krb5-libs x86_641.17-9.el8 CentOS-Base 851 k 2020-11-20T13:53:54.254 INFO:teuthology.orchestra.run.smithi116.stdout: krb5-workstation x86_641.17-9.el8 CentOS-Base 940 k 2020-11-20T13:53:54.255 INFO:teuthology.orchestra.run.smithi116.stdout: libkadm5 x86_641.17-9.el8 CentOS-Base 184 k 2020-11-20T13:53:54.255 INFO:teuthology.orchestra.run.smithi116.stdout: openssl x86_641:1.1.1c-2.el8_1.1 CentOS-Base 686 k 2020-11-20T13:53:54.255 INFO:teuthology.orchestra.run.smithi116.stdout: openssl-libs x86_641:1.1.1c-2.el8_1.1 CentOS-Base 1.5 M 2020-11-20T13:53:54.255 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T13:53:54.255 INFO:teuthology.orchestra.run.smithi116.stdout:Transaction Summary 2020-11-20T13:53:54.256 INFO:teuthology.orchestra.run.smithi116.stdout:================================================================================ 2020-11-20T13:53:54.256 INFO:teuthology.orchestra.run.smithi116.stdout:Install 126 Packages 2020-11-20T13:53:54.256 INFO:teuthology.orchestra.run.smithi116.stdout:Upgrade 2 Packages 2020-11-20T13:53:54.256 INFO:teuthology.orchestra.run.smithi116.stdout:Downgrade 5 Packages 2020-11-20T13:53:54.257 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T13:53:54.257 INFO:teuthology.orchestra.run.smithi116.stdout:Total download size: 185 M 2020-11-20T13:53:54.257 INFO:teuthology.orchestra.run.smithi116.stdout:Downloading Packages: 2020-11-20T13:53:54.271 INFO:teuthology.orchestra.run.smithi116.stdout:(1/133): libkadm5-1.17-9.el8.x86_64.rpm 5.9 MB/s | 184 kB 00:00 2020-11-20T13:53:54.280 INFO:teuthology.orchestra.run.smithi116.stdout:(2/133): openssl-1.1.1c-2.el8_1.1.x86_64.rpm 69 MB/s | 686 kB 00:00 2020-11-20T13:53:54.307 INFO:teuthology.orchestra.run.smithi116.stdout:(3/133): openssl-libs-1.1.1c-2.el8_1.1.x86_64.r 56 MB/s | 1.5 MB 00:00 2020-11-20T13:53:54.310 INFO:teuthology.orchestra.run.smithi116.stdout:(4/133): krb5-libs-1.17-9.el8.x86_64.rpm 12 MB/s | 851 kB 00:00 2020-11-20T13:53:54.317 INFO:teuthology.orchestra.run.smithi116.stdout:(5/133): krb5-workstation-1.17-9.el8.x86_64.rpm 12 MB/s | 940 kB 00:00 2020-11-20T13:53:54.325 INFO:teuthology.orchestra.run.smithi040.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2020-11-20T13:53:54.326 INFO:teuthology.orchestra.run.smithi040.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2020-11-20T13:53:54.329 INFO:teuthology.orchestra.run.smithi040.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2020-11-20T13:53:54.330 INFO:teuthology.orchestra.run.smithi040.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2020-11-20T13:53:54.331 INFO:teuthology.orchestra.run.smithi040.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2020-11-20T13:53:54.331 INFO:teuthology.orchestra.run.smithi040.stdout:Package xfsprogs-5.0.0-1.el8.x86_64 is already installed. 2020-11-20T13:53:54.407 INFO:teuthology.orchestra.run.smithi116.stdout:(6/133): ceph-15.2.5-588.ga7b62340.el8.x86_64.r 64 kB/s | 6.4 kB 00:00 2020-11-20T13:53:54.471 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2020-11-20T13:53:54.497 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2020-11-20T13:53:54.497 INFO:teuthology.orchestra.run.smithi040.stdout: Package ArchVersion Repository Size 2020-11-20T13:53:54.497 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2020-11-20T13:53:54.498 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2020-11-20T13:53:54.498 INFO:teuthology.orchestra.run.smithi040.stdout: ceph x86_642:15.2.5-588.ga7b62340.el8 ceph 6.4 k 2020-11-20T13:53:54.498 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-base x86_642:15.2.5-588.ga7b62340.el8 ceph 5.6 M 2020-11-20T13:53:54.498 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-fuse x86_642:15.2.5-588.ga7b62340.el8 ceph 577 k 2020-11-20T13:53:54.499 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-immutable-object-cache x86_642:15.2.5-588.ga7b62340.el8 ceph 205 k 2020-11-20T13:53:54.499 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr x86_642:15.2.5-588.ga7b62340.el8 ceph 1.4 M 2020-11-20T13:53:54.499 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-radosgw x86_642:15.2.5-588.ga7b62340.el8 ceph 10 M 2020-11-20T13:53:54.499 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-test x86_642:15.2.5-588.ga7b62340.el8 ceph 42 M 2020-11-20T13:53:54.499 INFO:teuthology.orchestra.run.smithi040.stdout: cephadm x86_642:15.2.5-588.ga7b62340.el8 ceph 57 k 2020-11-20T13:53:54.500 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs-devel x86_642:15.2.5-588.ga7b62340.el8 ceph 22 k 2020-11-20T13:53:54.500 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs2 x86_642:15.2.5-588.ga7b62340.el8 ceph 511 k 2020-11-20T13:53:54.500 INFO:teuthology.orchestra.run.smithi040.stdout: librados-devel x86_642:15.2.5-588.ga7b62340.el8 ceph 141 k 2020-11-20T13:53:54.500 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cephfs x86_642:15.2.5-588.ga7b62340.el8 ceph 163 k 2020-11-20T13:53:54.501 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rados x86_642:15.2.5-588.ga7b62340.el8 ceph 351 k 2020-11-20T13:53:54.501 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rbd x86_642:15.2.5-588.ga7b62340.el8 ceph 334 k 2020-11-20T13:53:54.501 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rgw x86_642:15.2.5-588.ga7b62340.el8 ceph 113 k 2020-11-20T13:53:54.501 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-fuse x86_642:15.2.5-588.ga7b62340.el8 ceph 72 k 2020-11-20T13:53:54.501 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-mirror x86_642:15.2.5-588.ga7b62340.el8 ceph 2.7 M 2020-11-20T13:53:54.502 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-nbd x86_642:15.2.5-588.ga7b62340.el8 ceph 141 k 2020-11-20T13:53:54.502 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-cephadm noarch2:15.2.5-588.ga7b62340.el8 ceph-noarch 65 k 2020-11-20T13:53:54.502 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-dashboard noarch2:15.2.5-588.ga7b62340.el8 ceph-noarch 3.5 M 2020-11-20T13:53:54.502 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-diskprediction-local noarch2:15.2.5-588.ga7b62340.el8 ceph-noarch 8.6 M 2020-11-20T13:53:54.503 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-rook noarch2:15.2.5-588.ga7b62340.el8 ceph-noarch 27 k 2020-11-20T13:53:54.503 INFO:teuthology.orchestra.run.smithi040.stdout: bison x86_643.0.4-10.el8 CentOS-AppStream 688 k 2020-11-20T13:53:54.503 INFO:teuthology.orchestra.run.smithi040.stdout: flex x86_642.6.1-9.el8 CentOS-AppStream 320 k 2020-11-20T13:53:54.503 INFO:teuthology.orchestra.run.smithi040.stdout: libtool x86_642.4.6-25.el8 CentOS-AppStream 709 k 2020-11-20T13:53:54.503 INFO:teuthology.orchestra.run.smithi040.stdout: elfutils-libelf-devel x86_640.176-5.el8 CentOS-Base 54 k 2020-11-20T13:53:54.504 INFO:teuthology.orchestra.run.smithi040.stdout: libaio-devel x86_640.3.112-1.el8 CentOS-Base 19 k 2020-11-20T13:53:54.504 INFO:teuthology.orchestra.run.smithi040.stdout: libuuid-devel x86_642.32.1-17.el8 CentOS-Base 95 k 2020-11-20T13:53:54.504 INFO:teuthology.orchestra.run.smithi040.stdout: openssl-devel x86_641:1.1.1c-2.el8_1.1 CentOS-Base 2.3 M 2020-11-20T13:53:54.504 INFO:teuthology.orchestra.run.smithi040.stdout: xfsprogs-devel x86_645.0.0-1.el8 CentOS-Base 69 k 2020-11-20T13:53:54.505 INFO:teuthology.orchestra.run.smithi040.stdout:Upgrading: 2020-11-20T13:53:54.505 INFO:teuthology.orchestra.run.smithi040.stdout: librados2 x86_642:15.2.5-588.ga7b62340.el8 ceph 3.6 M 2020-11-20T13:53:54.505 INFO:teuthology.orchestra.run.smithi040.stdout: librbd1 x86_642:15.2.5-588.ga7b62340.el8 ceph 1.9 M 2020-11-20T13:53:54.505 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2020-11-20T13:53:54.505 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-common x86_642:15.2.5-588.ga7b62340.el8 ceph 21 M 2020-11-20T13:53:54.506 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mds x86_642:15.2.5-588.ga7b62340.el8 ceph 2.0 M 2020-11-20T13:53:54.506 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mon x86_642:15.2.5-588.ga7b62340.el8 ceph 4.4 M 2020-11-20T13:53:54.506 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-osd x86_642:15.2.5-588.ga7b62340.el8 ceph 17 M 2020-11-20T13:53:54.506 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-selinux x86_642:15.2.5-588.ga7b62340.el8 ceph 25 k 2020-11-20T13:53:54.506 INFO:teuthology.orchestra.run.smithi040.stdout: libradosstriper1 x86_642:15.2.5-588.ga7b62340.el8 ceph 381 k 2020-11-20T13:53:54.507 INFO:teuthology.orchestra.run.smithi040.stdout: librgw2 x86_642:15.2.5-588.ga7b62340.el8 ceph 3.2 M 2020-11-20T13:53:54.507 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-argparse x86_642:15.2.5-588.ga7b62340.el8 ceph 40 k 2020-11-20T13:53:54.507 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-common x86_642:15.2.5-588.ga7b62340.el8 ceph 73 k 2020-11-20T13:53:54.507 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-grafana-dashboards noarch2:15.2.5-588.ga7b62340.el8 ceph-noarch 24 k 2020-11-20T13:53:54.507 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-modules-core noarch2:15.2.5-588.ga7b62340.el8 ceph-noarch 176 k 2020-11-20T13:53:54.508 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-prometheus-alerts noarch2:15.2.5-588.ga7b62340.el8 ceph-noarch 8.7 k 2020-11-20T13:53:54.508 INFO:teuthology.orchestra.run.smithi040.stdout: automake noarch1.16.1-6.el8 CentOS-AppStream 713 k 2020-11-20T13:53:54.508 INFO:teuthology.orchestra.run.smithi040.stdout: jq x86_641.5-12.el8 CentOS-AppStream 161 k 2020-11-20T13:53:54.508 INFO:teuthology.orchestra.run.smithi040.stdout: librdkafka x86_640.11.4-1.el8 CentOS-AppStream 353 k 2020-11-20T13:53:54.508 INFO:teuthology.orchestra.run.smithi040.stdout: lttng-ust x86_642.8.1-11.el8 CentOS-AppStream 259 k 2020-11-20T13:53:54.509 INFO:teuthology.orchestra.run.smithi040.stdout: oniguruma x86_646.8.2-1.el8 CentOS-AppStream 188 k 2020-11-20T13:53:54.509 INFO:teuthology.orchestra.run.smithi040.stdout: openblas x86_640.3.3-2.el8 CentOS-AppStream 4.3 M 2020-11-20T13:53:54.509 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-threads x86_640.3.3-2.el8 CentOS-AppStream 4.4 M 2020-11-20T13:53:54.509 INFO:teuthology.orchestra.run.smithi040.stdout: python3-babel noarch2.5.1-5.el8 CentOS-AppStream 4.8 M 2020-11-20T13:53:54.509 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jinja2 noarch2.10.1-2.el8_0 CentOS-AppStream 538 k 2020-11-20T13:53:54.510 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpatch noarch1.21-2.el8 CentOS-AppStream 27 k 2020-11-20T13:53:54.510 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpointer noarch1.10-11.el8 CentOS-AppStream 20 k 2020-11-20T13:53:54.510 INFO:teuthology.orchestra.run.smithi040.stdout: python3-lxml x86_644.2.3-1.el8 CentOS-AppStream 1.5 M 2020-11-20T13:53:54.510 INFO:teuthology.orchestra.run.smithi040.stdout: python3-mako noarch1.0.6-13.el8 CentOS-AppStream 157 k 2020-11-20T13:53:54.510 INFO:teuthology.orchestra.run.smithi040.stdout: python3-markupsafe x86_640.23-19.el8 CentOS-AppStream 39 k 2020-11-20T13:53:54.511 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy x86_641:1.14.3-9.el8 CentOS-AppStream 3.7 M 2020-11-20T13:53:54.511 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-f2py x86_641:1.14.3-9.el8 CentOS-AppStream 225 k 2020-11-20T13:53:54.511 INFO:teuthology.orchestra.run.smithi040.stdout: python3-prettytable noarch0.7.2-14.el8 CentOS-AppStream 44 k 2020-11-20T13:53:54.511 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1 noarch0.3.7-6.el8 CentOS-AppStream 126 k 2020-11-20T13:53:54.511 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-modules noarch0.3.7-6.el8 CentOS-AppStream 110 k 2020-11-20T13:53:54.512 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pytz noarch2017.2-9.el8 CentOS-AppStream 54 k 2020-11-20T13:53:54.512 INFO:teuthology.orchestra.run.smithi040.stdout: python3-scipy x86_641.0.0-20.module_el8.1.0+245+c39af44f CentOS-AppStream 15 M 2020-11-20T13:53:54.512 INFO:teuthology.orchestra.run.smithi040.stdout: python3-webencodings noarch0.5.1-6.el8 CentOS-AppStream 27 k 2020-11-20T13:53:54.512 INFO:teuthology.orchestra.run.smithi040.stdout: python3-werkzeug noarch0.12.2-4.el8 CentOS-AppStream 457 k 2020-11-20T13:53:54.512 INFO:teuthology.orchestra.run.smithi040.stdout: socat x86_641.7.3.2-6.el8 CentOS-AppStream 298 k 2020-11-20T13:53:54.513 INFO:teuthology.orchestra.run.smithi040.stdout: fuse x86_642.9.7-12.el8 CentOS-Base 82 k 2020-11-20T13:53:54.513 INFO:teuthology.orchestra.run.smithi040.stdout: fuse-common x86_643.2.1-12.el8 CentOS-Base 21 k 2020-11-20T13:53:54.513 INFO:teuthology.orchestra.run.smithi040.stdout: keyutils-libs-devel x86_641.5.10-6.el8 CentOS-Base 48 k 2020-11-20T13:53:54.513 INFO:teuthology.orchestra.run.smithi040.stdout: krb5-devel x86_641.17-9.el8 CentOS-Base 548 k 2020-11-20T13:53:54.513 INFO:teuthology.orchestra.run.smithi040.stdout: libcom_err-devel x86_641.44.6-3.el8 CentOS-Base 38 k 2020-11-20T13:53:54.514 INFO:teuthology.orchestra.run.smithi040.stdout: libgfortran x86_648.3.1-4.5.el8 CentOS-Base 638 k 2020-11-20T13:53:54.514 INFO:teuthology.orchestra.run.smithi040.stdout: libquadmath x86_648.3.1-4.5.el8 CentOS-Base 169 k 2020-11-20T13:53:54.514 INFO:teuthology.orchestra.run.smithi040.stdout: librabbitmq x86_640.9.0-1.el8 CentOS-Base 47 k 2020-11-20T13:53:54.514 INFO:teuthology.orchestra.run.smithi040.stdout: libselinux-devel x86_642.9-2.1.el8 CentOS-Base 199 k 2020-11-20T13:53:54.515 INFO:teuthology.orchestra.run.smithi040.stdout: libsepol-devel x86_642.9-1.el8 CentOS-Base 86 k 2020-11-20T13:53:54.515 INFO:teuthology.orchestra.run.smithi040.stdout: libverto-devel x86_640.3.0-5.el8 CentOS-Base 18 k 2020-11-20T13:53:54.515 INFO:teuthology.orchestra.run.smithi040.stdout: libxslt x86_641.1.32-3.el8 CentOS-Base 249 k 2020-11-20T13:53:54.515 INFO:teuthology.orchestra.run.smithi040.stdout: pcre2-devel x86_6410.32-1.el8 CentOS-Base 605 k 2020-11-20T13:53:54.515 INFO:teuthology.orchestra.run.smithi040.stdout: pcre2-utf16 x86_6410.32-1.el8 CentOS-Base 228 k 2020-11-20T13:53:54.516 INFO:teuthology.orchestra.run.smithi040.stdout: pcre2-utf32 x86_6410.32-1.el8 CentOS-Base 220 k 2020-11-20T13:53:54.516 INFO:teuthology.orchestra.run.smithi040.stdout: python3-chardet noarch3.0.4-7.el8 CentOS-Base 195 k 2020-11-20T13:53:54.516 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jwt noarch1.6.1-2.el8 CentOS-Base 43 k 2020-11-20T13:53:54.516 INFO:teuthology.orchestra.run.smithi040.stdout: python3-oauthlib noarch2.1.0-1.el8 CentOS-Base 155 k 2020-11-20T13:53:54.516 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pysocks noarch1.6.8-3.el8 CentOS-Base 34 k 2020-11-20T13:53:54.517 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests noarch2.20.0-2.1.el8_1 CentOS-Base 123 k 2020-11-20T13:53:54.517 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-oauthlib noarch1.0.0-1.el8 CentOS-Base 43 k 2020-11-20T13:53:54.517 INFO:teuthology.orchestra.run.smithi040.stdout: python3-urllib3 noarch1.24.2-2.el8 CentOS-Base 176 k 2020-11-20T13:53:54.517 INFO:teuthology.orchestra.run.smithi040.stdout: zlib-devel x86_641.2.11-10.el8 CentOS-Base 56 k 2020-11-20T13:53:54.517 INFO:teuthology.orchestra.run.smithi040.stdout: fmt x86_646.2.1-1.el8 epel 116 k 2020-11-20T13:53:54.518 INFO:teuthology.orchestra.run.smithi040.stdout: gperftools-libs x86_642.7-6.el8 epel 315 k 2020-11-20T13:53:54.518 INFO:teuthology.orchestra.run.smithi040.stdout: leveldb x86_641.22-1.el8 epel 181 k 2020-11-20T13:53:54.518 INFO:teuthology.orchestra.run.smithi040.stdout: liboath x86_642.6.2-3.el8 epel 59 k 2020-11-20T13:53:54.518 INFO:teuthology.orchestra.run.smithi040.stdout: libunwind x86_641.3.1-3.el8 epel 75 k 2020-11-20T13:53:54.518 INFO:teuthology.orchestra.run.smithi040.stdout: python3-apipkg noarch1.5-6.el8 epel 21 k 2020-11-20T13:53:54.519 INFO:teuthology.orchestra.run.smithi040.stdout: python3-bcrypt x86_643.1.6-2.el8.1 epel 44 k 2020-11-20T13:53:54.519 INFO:teuthology.orchestra.run.smithi040.stdout: python3-beautifulsoup4 noarch4.6.3-2.el8.1 epel 185 k 2020-11-20T13:53:54.519 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cachetools noarch3.1.1-4.el8 epel 33 k 2020-11-20T13:53:54.519 INFO:teuthology.orchestra.run.smithi040.stdout: python3-certifi noarch2018.10.15-7.el8 epel 16 k 2020-11-20T13:53:54.519 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cheroot noarch8.2.1-1.el8 epel 145 k 2020-11-20T13:53:54.520 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cherrypy noarch18.4.0-1.el8 epel 384 k 2020-11-20T13:53:54.520 INFO:teuthology.orchestra.run.smithi040.stdout: python3-execnet noarch1.7.1-1.el8 epel 355 k 2020-11-20T13:53:54.520 INFO:teuthology.orchestra.run.smithi040.stdout: python3-google-auth noarch1:1.1.1-10.el8 epel 81 k 2020-11-20T13:53:54.520 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco noarch6.2-6.el8 epel 11 k 2020-11-20T13:53:54.520 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-functools noarch2.0-4.el8 epel 18 k 2020-11-20T13:53:54.521 INFO:teuthology.orchestra.run.smithi040.stdout: python3-kubernetes noarch11.0.0-2.el8 epel 1.2 M 2020-11-20T13:53:54.521 INFO:teuthology.orchestra.run.smithi040.stdout: python3-logutils noarch0.3.5-11.el8 epel 49 k 2020-11-20T13:53:54.521 INFO:teuthology.orchestra.run.smithi040.stdout: python3-more-itertools noarch7.2.0-3.el8 epel 59 k 2020-11-20T13:53:54.521 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pecan noarch1.3.2-9.el8 epel 283 k 2020-11-20T13:53:54.521 INFO:teuthology.orchestra.run.smithi040.stdout: python3-portend noarch2.6-1.el8 epel 16 k 2020-11-20T13:53:54.522 INFO:teuthology.orchestra.run.smithi040.stdout: python3-remoto noarch1.1.4-4.el8 epel 48 k 2020-11-20T13:53:54.522 INFO:teuthology.orchestra.run.smithi040.stdout: python3-repoze-lru noarch0.7-6.el8 epel 34 k 2020-11-20T13:53:54.522 INFO:teuthology.orchestra.run.smithi040.stdout: python3-routes noarch2.4.1-12.el8 epel 196 k 2020-11-20T13:53:54.522 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rsa noarch3.4.2-15.el8 epel 73 k 2020-11-20T13:53:54.522 INFO:teuthology.orchestra.run.smithi040.stdout: python3-simplegeneric noarch0.8.1-17.el8 epel 19 k 2020-11-20T13:53:54.523 INFO:teuthology.orchestra.run.smithi040.stdout: python3-singledispatch noarch3.4.0.3-18.el8 epel 24 k 2020-11-20T13:53:54.523 INFO:teuthology.orchestra.run.smithi040.stdout: python3-tempora noarch1.14.1-5.el8 epel 29 k 2020-11-20T13:53:54.523 INFO:teuthology.orchestra.run.smithi040.stdout: python3-trustme noarch0.6.0-4.el8 epel 27 k 2020-11-20T13:53:54.523 INFO:teuthology.orchestra.run.smithi040.stdout: python3-waitress noarch1.2.1-2.el8.1 epel 241 k 2020-11-20T13:53:54.523 INFO:teuthology.orchestra.run.smithi040.stdout: python3-webob noarch1.8.5-1.el8.1 epel 251 k 2020-11-20T13:53:54.524 INFO:teuthology.orchestra.run.smithi040.stdout: python3-websocket-client noarch0.56.0-5.el8 epel 61 k 2020-11-20T13:53:54.524 INFO:teuthology.orchestra.run.smithi040.stdout: python3-webtest noarch2.0.33-1.el8 epel 85 k 2020-11-20T13:53:54.524 INFO:teuthology.orchestra.run.smithi040.stdout: python3-zc-lockfile noarch2.0-2.el8 epel 23 k 2020-11-20T13:53:54.524 INFO:teuthology.orchestra.run.smithi040.stdout: xmlstarlet x86_641.6.1-11.el8 epel 69 k 2020-11-20T13:53:54.524 INFO:teuthology.orchestra.run.smithi040.stdout:Installing weak dependencies: 2020-11-20T13:53:54.525 INFO:teuthology.orchestra.run.smithi040.stdout: python3-html5lib noarch1:0.999999999-6.el8 CentOS-AppStream 214 k 2020-11-20T13:53:54.525 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cssselect noarch0.9.2-10.el8 epel 40 k 2020-11-20T13:53:54.525 INFO:teuthology.orchestra.run.smithi040.stdout:Downgrading: 2020-11-20T13:53:54.525 INFO:teuthology.orchestra.run.smithi040.stdout: krb5-libs x86_641.17-9.el8 CentOS-Base 851 k 2020-11-20T13:53:54.525 INFO:teuthology.orchestra.run.smithi040.stdout: krb5-workstation x86_641.17-9.el8 CentOS-Base 940 k 2020-11-20T13:53:54.526 INFO:teuthology.orchestra.run.smithi040.stdout: libkadm5 x86_641.17-9.el8 CentOS-Base 184 k 2020-11-20T13:53:54.526 INFO:teuthology.orchestra.run.smithi040.stdout: openssl x86_641:1.1.1c-2.el8_1.1 CentOS-Base 686 k 2020-11-20T13:53:54.526 INFO:teuthology.orchestra.run.smithi040.stdout: openssl-libs x86_641:1.1.1c-2.el8_1.1 CentOS-Base 1.5 M 2020-11-20T13:53:54.526 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:53:54.527 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2020-11-20T13:53:54.527 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2020-11-20T13:53:54.527 INFO:teuthology.orchestra.run.smithi040.stdout:Install 126 Packages 2020-11-20T13:53:54.527 INFO:teuthology.orchestra.run.smithi040.stdout:Upgrade 2 Packages 2020-11-20T13:53:54.527 INFO:teuthology.orchestra.run.smithi040.stdout:Downgrade 5 Packages 2020-11-20T13:53:54.528 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:53:54.528 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 185 M 2020-11-20T13:53:54.529 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2020-11-20T13:53:54.547 INFO:teuthology.orchestra.run.smithi116.stdout:(7/133): ceph-fuse-15.2.5-588.ga7b62340.el8.x86 4.0 MB/s | 577 kB 00:00 2020-11-20T13:53:54.560 INFO:teuthology.orchestra.run.smithi040.stdout:(1/133): libkadm5-1.17-9.el8.x86_64.rpm 4.0 MB/s | 184 kB 00:00 2020-11-20T13:53:54.562 INFO:teuthology.orchestra.run.smithi040.stdout:(2/133): krb5-libs-1.17-9.el8.x86_64.rpm 17 MB/s | 851 kB 00:00 2020-11-20T13:53:54.570 INFO:teuthology.orchestra.run.smithi040.stdout:(3/133): krb5-workstation-1.17-9.el8.x86_64.rpm 17 MB/s | 940 kB 00:00 2020-11-20T13:53:54.576 INFO:teuthology.orchestra.run.smithi116.stdout:(8/133): ceph-immutable-object-cache-15.2.5-588 6.9 MB/s | 205 kB 00:00 2020-11-20T13:53:54.581 INFO:teuthology.orchestra.run.smithi040.stdout:(4/133): openssl-1.1.1c-2.el8_1.1.x86_64.rpm 32 MB/s | 686 kB 00:00 2020-11-20T13:53:54.592 INFO:teuthology.orchestra.run.smithi040.stdout:(5/133): openssl-libs-1.1.1c-2.el8_1.1.x86_64.r 50 MB/s | 1.5 MB 00:00 2020-11-20T13:53:54.712 INFO:teuthology.orchestra.run.smithi040.stdout:(6/133): ceph-15.2.5-588.ga7b62340.el8.x86_64.r 45 kB/s | 6.4 kB 00:00 2020-11-20T13:53:54.837 INFO:teuthology.orchestra.run.smithi116.stdout:(9/133): ceph-mds-15.2.5-588.ga7b62340.el8.x86_ 7.8 MB/s | 2.0 MB 00:00 2020-11-20T13:53:55.040 INFO:teuthology.orchestra.run.smithi116.stdout:(10/133): ceph-base-15.2.5-588.ga7b62340.el8.x8 7.6 MB/s | 5.6 MB 00:00 2020-11-20T13:53:55.044 INFO:teuthology.orchestra.run.smithi116.stdout:(11/133): ceph-mgr-15.2.5-588.ga7b62340.el8.x86 6.8 MB/s | 1.4 MB 00:00 2020-11-20T13:53:55.255 INFO:teuthology.orchestra.run.smithi040.stdout:(7/133): ceph-fuse-15.2.5-588.ga7b62340.el8.x86 1.0 MB/s | 577 kB 00:00 2020-11-20T13:53:55.480 INFO:teuthology.orchestra.run.smithi040.stdout:(8/133): ceph-immutable-object-cache-15.2.5-588 911 kB/s | 205 kB 00:00 2020-11-20T13:53:55.521 INFO:teuthology.orchestra.run.smithi116.stdout:(12/133): ceph-mon-15.2.5-588.ga7b62340.el8.x86 9.1 MB/s | 4.4 MB 00:00 2020-11-20T13:53:56.691 INFO:teuthology.orchestra.run.smithi040.stdout:(9/133): ceph-mds-15.2.5-588.ga7b62340.el8.x86_ 1.7 MB/s | 2.0 MB 00:01 2020-11-20T13:53:56.834 INFO:teuthology.orchestra.run.smithi116.stdout:(13/133): ceph-common-15.2.5-588.ga7b62340.el8. 8.3 MB/s | 21 MB 00:02 2020-11-20T13:53:56.912 INFO:teuthology.orchestra.run.smithi116.stdout:(14/133): ceph-radosgw-15.2.5-588.ga7b62340.el8 7.3 MB/s | 10 MB 00:01 2020-11-20T13:53:56.912 INFO:teuthology.orchestra.run.smithi116.stdout:(15/133): ceph-selinux-15.2.5-588.ga7b62340.el8 320 kB/s | 25 kB 00:00 2020-11-20T13:53:56.916 INFO:teuthology.orchestra.run.smithi040.stdout:(10/133): ceph-mgr-15.2.5-588.ga7b62340.el8.x86 6.3 MB/s | 1.4 MB 00:00 2020-11-20T13:53:56.973 INFO:teuthology.orchestra.run.smithi116.stdout:(16/133): cephadm-15.2.5-588.ga7b62340.el8.x86_ 944 kB/s | 57 kB 00:00 2020-11-20T13:53:57.030 INFO:teuthology.orchestra.run.smithi116.stdout:(17/133): libcephfs-devel-15.2.5-588.ga7b62340. 381 kB/s | 22 kB 00:00 2020-11-20T13:53:57.107 INFO:teuthology.orchestra.run.smithi116.stdout:(18/133): libcephfs2-15.2.5-588.ga7b62340.el8.x 6.5 MB/s | 511 kB 00:00 2020-11-20T13:53:57.221 INFO:teuthology.orchestra.run.smithi116.stdout:(19/133): ceph-osd-15.2.5-588.ga7b62340.el8.x86 7.7 MB/s | 17 MB 00:02 2020-11-20T13:53:57.222 INFO:teuthology.orchestra.run.smithi116.stdout:(20/133): librados-devel-15.2.5-588.ga7b62340.e 1.2 MB/s | 141 kB 00:00 2020-11-20T13:53:57.276 INFO:teuthology.orchestra.run.smithi116.stdout:(21/133): libradosstriper1-15.2.5-588.ga7b62340 6.9 MB/s | 381 kB 00:00 2020-11-20T13:53:57.341 INFO:teuthology.orchestra.run.smithi116.stdout:(22/133): python3-ceph-argparse-15.2.5-588.ga7b 611 kB/s | 40 kB 00:00 2020-11-20T13:53:57.409 INFO:teuthology.orchestra.run.smithi116.stdout:(23/133): python3-ceph-common-15.2.5-588.ga7b62 1.1 MB/s | 73 kB 00:00 2020-11-20T13:53:57.481 INFO:teuthology.orchestra.run.smithi116.stdout:(24/133): python3-cephfs-15.2.5-588.ga7b62340.e 2.2 MB/s | 163 kB 00:00 2020-11-20T13:53:57.561 INFO:teuthology.orchestra.run.smithi116.stdout:(25/133): python3-rados-15.2.5-588.ga7b62340.el 4.3 MB/s | 351 kB 00:00 2020-11-20T13:53:57.587 INFO:teuthology.orchestra.run.smithi116.stdout:(26/133): librgw2-15.2.5-588.ga7b62340.el8.x86_ 8.9 MB/s | 3.2 MB 00:00 2020-11-20T13:53:57.633 INFO:teuthology.orchestra.run.smithi116.stdout:(27/133): python3-rbd-15.2.5-588.ga7b62340.el8. 4.5 MB/s | 334 kB 00:00 2020-11-20T13:53:57.648 INFO:teuthology.orchestra.run.smithi116.stdout:(28/133): python3-rgw-15.2.5-588.ga7b62340.el8. 1.8 MB/s | 113 kB 00:00 2020-11-20T13:53:57.660 INFO:teuthology.orchestra.run.smithi040.stdout:(11/133): ceph-mon-15.2.5-588.ga7b62340.el8.x86 5.9 MB/s | 4.4 MB 00:00 2020-11-20T13:53:57.679 INFO:teuthology.orchestra.run.smithi116.stdout:(29/133): rbd-fuse-15.2.5-588.ga7b62340.el8.x86 1.5 MB/s | 72 kB 00:00 2020-11-20T13:53:57.754 INFO:teuthology.orchestra.run.smithi116.stdout:(30/133): rbd-nbd-15.2.5-588.ga7b62340.el8.x86_ 1.9 MB/s | 141 kB 00:00 2020-11-20T13:53:57.825 INFO:teuthology.orchestra.run.smithi116.stdout:(31/133): ceph-grafana-dashboards-15.2.5-588.ga 339 kB/s | 24 kB 00:00 2020-11-20T13:53:57.899 INFO:teuthology.orchestra.run.smithi116.stdout:(32/133): ceph-mgr-cephadm-15.2.5-588.ga7b62340 882 kB/s | 65 kB 00:00 2020-11-20T13:53:57.978 INFO:teuthology.orchestra.run.smithi116.stdout:(33/133): rbd-mirror-15.2.5-588.ga7b62340.el8.x 8.3 MB/s | 2.7 MB 00:00 2020-11-20T13:53:58.556 INFO:teuthology.orchestra.run.smithi040.stdout:(12/133): ceph-base-15.2.5-588.ga7b62340.el8.x8 1.4 MB/s | 5.6 MB 00:03 2020-11-20T13:53:58.633 INFO:teuthology.orchestra.run.smithi116.stdout:(34/133): ceph-mgr-dashboard-15.2.5-588.ga7b623 4.8 MB/s | 3.5 MB 00:00 2020-11-20T13:53:58.698 INFO:teuthology.orchestra.run.smithi116.stdout:(35/133): ceph-mgr-modules-core-15.2.5-588.ga7b 2.7 MB/s | 176 kB 00:00 2020-11-20T13:53:58.757 INFO:teuthology.orchestra.run.smithi116.stdout:(36/133): ceph-mgr-rook-15.2.5-588.ga7b62340.el 459 kB/s | 27 kB 00:00 2020-11-20T13:53:58.816 INFO:teuthology.orchestra.run.smithi116.stdout:(37/133): ceph-prometheus-alerts-15.2.5-588.ga7 148 kB/s | 8.7 kB 00:00 2020-11-20T13:53:58.827 INFO:teuthology.orchestra.run.smithi116.stdout:(38/133): automake-1.16.1-6.el8.noarch.rpm 68 MB/s | 713 kB 00:00 2020-11-20T13:53:58.835 INFO:teuthology.orchestra.run.smithi116.stdout:(39/133): bison-3.0.4-10.el8.x86_64.rpm 78 MB/s | 688 kB 00:00 2020-11-20T13:53:58.840 INFO:teuthology.orchestra.run.smithi116.stdout:(40/133): flex-2.6.1-9.el8.x86_64.rpm 68 MB/s | 320 kB 00:00 2020-11-20T13:53:58.843 INFO:teuthology.orchestra.run.smithi116.stdout:(41/133): jq-1.5-12.el8.x86_64.rpm 58 MB/s | 161 kB 00:00 2020-11-20T13:53:58.848 INFO:teuthology.orchestra.run.smithi116.stdout:(42/133): librdkafka-0.11.4-1.el8.x86_64.rpm 71 MB/s | 353 kB 00:00 2020-11-20T13:53:58.857 INFO:teuthology.orchestra.run.smithi116.stdout:(43/133): libtool-2.4.6-25.el8.x86_64.rpm 73 MB/s | 709 kB 00:00 2020-11-20T13:53:58.862 INFO:teuthology.orchestra.run.smithi116.stdout:(44/133): lttng-ust-2.8.1-11.el8.x86_64.rpm 67 MB/s | 259 kB 00:00 2020-11-20T13:53:58.865 INFO:teuthology.orchestra.run.smithi116.stdout:(45/133): oniguruma-6.8.2-1.el8.x86_64.rpm 60 MB/s | 188 kB 00:00 2020-11-20T13:53:58.918 INFO:teuthology.orchestra.run.smithi116.stdout:(46/133): openblas-0.3.3-2.el8.x86_64.rpm 81 MB/s | 4.3 MB 00:00 2020-11-20T13:53:58.970 INFO:teuthology.orchestra.run.smithi116.stdout:(47/133): openblas-threads-0.3.3-2.el8.x86_64.r 85 MB/s | 4.4 MB 00:00 2020-11-20T13:53:59.026 INFO:teuthology.orchestra.run.smithi116.stdout:(48/133): python3-babel-2.5.1-5.el8.noarch.rpm 86 MB/s | 4.8 MB 00:00 2020-11-20T13:53:59.029 INFO:teuthology.orchestra.run.smithi116.stdout:(49/133): python3-html5lib-0.999999999-6.el8.no 60 MB/s | 214 kB 00:00 2020-11-20T13:53:59.036 INFO:teuthology.orchestra.run.smithi116.stdout:(50/133): python3-jinja2-2.10.1-2.el8_0.noarch. 77 MB/s | 538 kB 00:00 2020-11-20T13:53:59.037 INFO:teuthology.orchestra.run.smithi116.stdout:(51/133): python3-jsonpatch-1.21-2.el8.noarch.r 25 MB/s | 27 kB 00:00 2020-11-20T13:53:59.039 INFO:teuthology.orchestra.run.smithi116.stdout:(52/133): python3-jsonpointer-1.10-11.el8.noarc 17 MB/s | 20 kB 00:00 2020-11-20T13:53:59.057 INFO:teuthology.orchestra.run.smithi116.stdout:(53/133): python3-lxml-4.2.3-1.el8.x86_64.rpm 83 MB/s | 1.5 MB 00:00 2020-11-20T13:53:59.060 INFO:teuthology.orchestra.run.smithi116.stdout:(54/133): python3-mako-1.0.6-13.el8.noarch.rpm 55 MB/s | 157 kB 00:00 2020-11-20T13:53:59.062 INFO:teuthology.orchestra.run.smithi116.stdout:(55/133): python3-markupsafe-0.23-19.el8.x86_64 29 MB/s | 39 kB 00:00 2020-11-20T13:53:59.107 INFO:teuthology.orchestra.run.smithi116.stdout:(56/133): python3-numpy-1.14.3-9.el8.x86_64.rpm 83 MB/s | 3.7 MB 00:00 2020-11-20T13:53:59.111 INFO:teuthology.orchestra.run.smithi116.stdout:(57/133): python3-numpy-f2py-1.14.3-9.el8.x86_6 60 MB/s | 225 kB 00:00 2020-11-20T13:53:59.209 INFO:teuthology.orchestra.run.smithi116.stdout:(58/133): ceph-mgr-diskprediction-local-15.2.5- 7.0 MB/s | 8.6 MB 00:01 2020-11-20T13:53:59.210 INFO:teuthology.orchestra.run.smithi116.stdout:(59/133): python3-prettytable-0.7.2-14.el8.noar 445 kB/s | 44 kB 00:00 2020-11-20T13:53:59.213 INFO:teuthology.orchestra.run.smithi116.stdout:(60/133): python3-pyasn1-modules-0.3.7-6.el8.no 36 MB/s | 110 kB 00:00 2020-11-20T13:53:59.213 INFO:teuthology.orchestra.run.smithi116.stdout:(61/133): python3-pyasn1-0.3.7-6.el8.noarch.rpm 28 MB/s | 126 kB 00:00 2020-11-20T13:53:59.214 INFO:teuthology.orchestra.run.smithi116.stdout:(62/133): python3-pytz-2017.2-9.el8.noarch.rpm 39 MB/s | 54 kB 00:00 2020-11-20T13:53:59.218 INFO:teuthology.orchestra.run.smithi116.stdout:(63/133): python3-webencodings-0.5.1-6.el8.noar 7.7 MB/s | 27 kB 00:00 2020-11-20T13:53:59.225 INFO:teuthology.orchestra.run.smithi116.stdout:(64/133): python3-werkzeug-0.12.2-4.el8.noarch. 66 MB/s | 457 kB 00:00 2020-11-20T13:53:59.230 INFO:teuthology.orchestra.run.smithi116.stdout:(65/133): socat-1.7.3.2-6.el8.x86_64.rpm 61 MB/s | 298 kB 00:00 2020-11-20T13:53:59.232 INFO:teuthology.orchestra.run.smithi116.stdout:(66/133): elfutils-libelf-devel-0.176-5.el8.x86 31 MB/s | 54 kB 00:00 2020-11-20T13:53:59.234 INFO:teuthology.orchestra.run.smithi116.stdout:(67/133): fuse-2.9.7-12.el8.x86_64.rpm 38 MB/s | 82 kB 00:00 2020-11-20T13:53:59.235 INFO:teuthology.orchestra.run.smithi116.stdout:(68/133): fuse-common-3.2.1-12.el8.x86_64.rpm 17 MB/s | 21 kB 00:00 2020-11-20T13:53:59.237 INFO:teuthology.orchestra.run.smithi116.stdout:(69/133): keyutils-libs-devel-1.5.10-6.el8.x86_ 29 MB/s | 48 kB 00:00 2020-11-20T13:53:59.245 INFO:teuthology.orchestra.run.smithi116.stdout:(70/133): krb5-devel-1.17-9.el8.x86_64.rpm 68 MB/s | 548 kB 00:00 2020-11-20T13:53:59.247 INFO:teuthology.orchestra.run.smithi116.stdout:(71/133): libaio-devel-0.3.112-1.el8.x86_64.rpm 14 MB/s | 19 kB 00:00 2020-11-20T13:53:59.249 INFO:teuthology.orchestra.run.smithi116.stdout:(72/133): libcom_err-devel-1.44.6-3.el8.x86_64. 28 MB/s | 38 kB 00:00 2020-11-20T13:53:59.258 INFO:teuthology.orchestra.run.smithi116.stdout:(73/133): libgfortran-8.3.1-4.5.el8.x86_64.rpm 69 MB/s | 638 kB 00:00 2020-11-20T13:53:59.261 INFO:teuthology.orchestra.run.smithi116.stdout:(74/133): libquadmath-8.3.1-4.5.el8.x86_64.rpm 50 MB/s | 169 kB 00:00 2020-11-20T13:53:59.263 INFO:teuthology.orchestra.run.smithi116.stdout:(75/133): librabbitmq-0.9.0-1.el8.x86_64.rpm 28 MB/s | 47 kB 00:00 2020-11-20T13:53:59.267 INFO:teuthology.orchestra.run.smithi116.stdout:(76/133): libselinux-devel-2.9-2.1.el8.x86_64.r 53 MB/s | 199 kB 00:00 2020-11-20T13:53:59.269 INFO:teuthology.orchestra.run.smithi116.stdout:(77/133): libsepol-devel-2.9-1.el8.x86_64.rpm 35 MB/s | 86 kB 00:00 2020-11-20T13:53:59.272 INFO:teuthology.orchestra.run.smithi116.stdout:(78/133): libuuid-devel-2.32.1-17.el8.x86_64.rp 36 MB/s | 95 kB 00:00 2020-11-20T13:53:59.273 INFO:teuthology.orchestra.run.smithi116.stdout:(79/133): libverto-devel-0.3.0-5.el8.x86_64.rpm 13 MB/s | 18 kB 00:00 2020-11-20T13:53:59.278 INFO:teuthology.orchestra.run.smithi116.stdout:(80/133): libxslt-1.1.32-3.el8.x86_64.rpm 53 MB/s | 249 kB 00:00 2020-11-20T13:53:59.309 INFO:teuthology.orchestra.run.smithi116.stdout:(81/133): openssl-devel-1.1.1c-2.el8_1.1.x86_64 76 MB/s | 2.3 MB 00:00 2020-11-20T13:53:59.318 INFO:teuthology.orchestra.run.smithi116.stdout:(82/133): pcre2-devel-10.32-1.el8.x86_64.rpm 64 MB/s | 605 kB 00:00 2020-11-20T13:53:59.323 INFO:teuthology.orchestra.run.smithi116.stdout:(83/133): pcre2-utf16-10.32-1.el8.x86_64.rpm 50 MB/s | 228 kB 00:00 2020-11-20T13:53:59.327 INFO:teuthology.orchestra.run.smithi116.stdout:(84/133): pcre2-utf32-10.32-1.el8.x86_64.rpm 49 MB/s | 220 kB 00:00 2020-11-20T13:53:59.331 INFO:teuthology.orchestra.run.smithi116.stdout:(85/133): python3-chardet-3.0.4-7.el8.noarch.rp 47 MB/s | 195 kB 00:00 2020-11-20T13:53:59.333 INFO:teuthology.orchestra.run.smithi116.stdout:(86/133): python3-jwt-1.6.1-2.el8.noarch.rpm 25 MB/s | 43 kB 00:00 2020-11-20T13:53:59.337 INFO:teuthology.orchestra.run.smithi116.stdout:(87/133): python3-oauthlib-2.1.0-1.el8.noarch.r 44 MB/s | 155 kB 00:00 2020-11-20T13:53:59.339 INFO:teuthology.orchestra.run.smithi116.stdout:(88/133): python3-pysocks-1.6.8-3.el8.noarch.rp 20 MB/s | 34 kB 00:00 2020-11-20T13:53:59.342 INFO:teuthology.orchestra.run.smithi116.stdout:(89/133): python3-requests-2.20.0-2.1.el8_1.noa 40 MB/s | 123 kB 00:00 2020-11-20T13:53:59.344 INFO:teuthology.orchestra.run.smithi116.stdout:(90/133): python3-requests-oauthlib-1.0.0-1.el8 24 MB/s | 43 kB 00:00 2020-11-20T13:53:59.348 INFO:teuthology.orchestra.run.smithi116.stdout:(91/133): python3-urllib3-1.24.2-2.el8.noarch.r 47 MB/s | 176 kB 00:00 2020-11-20T13:53:59.350 INFO:teuthology.orchestra.run.smithi116.stdout:(92/133): xfsprogs-devel-5.0.0-1.el8.x86_64.rpm 29 MB/s | 69 kB 00:00 2020-11-20T13:53:59.352 INFO:teuthology.orchestra.run.smithi116.stdout:(93/133): zlib-devel-1.2.11-10.el8.x86_64.rpm 28 MB/s | 56 kB 00:00 2020-11-20T13:53:59.527 INFO:teuthology.orchestra.run.smithi116.stdout:(94/133): python3-scipy-1.0.0-20.module_el8.1.0 47 MB/s | 15 MB 00:00 2020-11-20T13:53:59.527 INFO:teuthology.orchestra.run.smithi116.stdout:(95/133): fmt-6.2.1-1.el8.x86_64.rpm 666 kB/s | 116 kB 00:00 2020-11-20T13:53:59.560 INFO:teuthology.orchestra.run.smithi116.stdout:(96/133): leveldb-1.22-1.el8.x86_64.rpm 5.4 MB/s | 181 kB 00:00 2020-11-20T13:53:59.577 INFO:teuthology.orchestra.run.smithi116.stdout:(97/133): liboath-2.6.2-3.el8.x86_64.rpm 3.5 MB/s | 59 kB 00:00 2020-11-20T13:53:59.594 INFO:teuthology.orchestra.run.smithi116.stdout:(98/133): libunwind-1.3.1-3.el8.x86_64.rpm 4.4 MB/s | 75 kB 00:00 2020-11-20T13:53:59.610 INFO:teuthology.orchestra.run.smithi116.stdout:(99/133): python3-apipkg-1.5-6.el8.noarch.rpm 1.3 MB/s | 21 kB 00:00 2020-11-20T13:53:59.628 INFO:teuthology.orchestra.run.smithi116.stdout:(100/133): python3-bcrypt-3.1.6-2.el8.1.x86_64. 2.4 MB/s | 44 kB 00:00 2020-11-20T13:53:59.643 INFO:teuthology.orchestra.run.smithi116.stdout:(101/133): gperftools-libs-2.7-6.el8.x86_64.rpm 2.7 MB/s | 315 kB 00:00 2020-11-20T13:53:59.646 INFO:teuthology.orchestra.run.smithi116.stdout:(102/133): python3-beautifulsoup4-4.6.3-2.el8.1 10 MB/s | 185 kB 00:00 2020-11-20T13:53:59.662 INFO:teuthology.orchestra.run.smithi116.stdout:(103/133): python3-certifi-2018.10.15-7.el8.noa 1.0 MB/s | 16 kB 00:00 2020-11-20T13:53:59.663 INFO:teuthology.orchestra.run.smithi116.stdout:(104/133): python3-cachetools-3.1.1-4.el8.noarc 1.7 MB/s | 33 kB 00:00 2020-11-20T13:53:59.679 INFO:teuthology.orchestra.run.smithi116.stdout:(105/133): python3-cheroot-8.2.1-1.el8.noarch.r 8.3 MB/s | 145 kB 00:00 2020-11-20T13:53:59.696 INFO:teuthology.orchestra.run.smithi116.stdout:(106/133): python3-cssselect-0.9.2-10.el8.noarc 2.4 MB/s | 40 kB 00:00 2020-11-20T13:53:59.702 INFO:teuthology.orchestra.run.smithi116.stdout:(107/133): python3-cherrypy-18.4.0-1.el8.noarch 9.5 MB/s | 384 kB 00:00 2020-11-20T13:53:59.714 INFO:teuthology.orchestra.run.smithi116.stdout:(108/133): python3-execnet-1.7.1-1.el8.noarch.r 19 MB/s | 355 kB 00:00 2020-11-20T13:53:59.723 INFO:teuthology.orchestra.run.smithi116.stdout:(109/133): python3-google-auth-1.1.1-10.el8.noa 4.0 MB/s | 81 kB 00:00 2020-11-20T13:53:59.724 INFO:teuthology.orchestra.run.smithi040.stdout:(13/133): ceph-osd-15.2.5-588.ga7b62340.el8.x86 8.1 MB/s | 17 MB 00:02 2020-11-20T13:53:59.731 INFO:teuthology.orchestra.run.smithi116.stdout:(110/133): python3-jaraco-6.2-6.el8.noarch.rpm 685 kB/s | 11 kB 00:00 2020-11-20T13:53:59.742 INFO:teuthology.orchestra.run.smithi116.stdout:(111/133): python3-jaraco-functools-2.0-4.el8.n 900 kB/s | 18 kB 00:00 2020-11-20T13:53:59.750 INFO:teuthology.orchestra.run.smithi040.stdout:(14/133): ceph-selinux-15.2.5-588.ga7b62340.el8 1.0 MB/s | 25 kB 00:00 2020-11-20T13:53:59.763 INFO:teuthology.orchestra.run.smithi116.stdout:(112/133): python3-logutils-0.3.5-11.el8.noarch 2.4 MB/s | 49 kB 00:00 2020-11-20T13:53:59.782 INFO:teuthology.orchestra.run.smithi116.stdout:(113/133): python3-more-itertools-7.2.0-3.el8.n 2.9 MB/s | 59 kB 00:00 2020-11-20T13:53:59.797 INFO:teuthology.orchestra.run.smithi116.stdout:(114/133): python3-kubernetes-11.0.0-2.el8.noar 18 MB/s | 1.2 MB 00:00 2020-11-20T13:53:59.804 INFO:teuthology.orchestra.run.smithi116.stdout:(115/133): python3-pecan-1.3.2-9.el8.noarch.rpm 13 MB/s | 283 kB 00:00 2020-11-20T13:53:59.814 INFO:teuthology.orchestra.run.smithi116.stdout:(116/133): python3-portend-2.6-1.el8.noarch.rpm 966 kB/s | 16 kB 00:00 2020-11-20T13:53:59.824 INFO:teuthology.orchestra.run.smithi116.stdout:(117/133): python3-remoto-1.1.4-4.el8.noarch.rp 2.4 MB/s | 48 kB 00:00 2020-11-20T13:53:59.831 INFO:teuthology.orchestra.run.smithi116.stdout:(118/133): python3-repoze-lru-0.7-6.el8.noarch. 2.0 MB/s | 34 kB 00:00 2020-11-20T13:53:59.845 INFO:teuthology.orchestra.run.smithi116.stdout:(119/133): python3-routes-2.4.1-12.el8.noarch.r 9.2 MB/s | 196 kB 00:00 2020-11-20T13:53:59.847 INFO:teuthology.orchestra.run.smithi116.stdout:(120/133): python3-rsa-3.4.2-15.el8.noarch.rpm 4.3 MB/s | 73 kB 00:00 2020-11-20T13:53:59.864 INFO:teuthology.orchestra.run.smithi116.stdout:(121/133): python3-singledispatch-3.4.0.3-18.el 1.4 MB/s | 24 kB 00:00 2020-11-20T13:53:59.865 INFO:teuthology.orchestra.run.smithi116.stdout:(122/133): python3-simplegeneric-0.8.1-17.el8.n 948 kB/s | 19 kB 00:00 2020-11-20T13:53:59.881 INFO:teuthology.orchestra.run.smithi116.stdout:(123/133): python3-tempora-1.14.1-5.el8.noarch. 1.7 MB/s | 29 kB 00:00 2020-11-20T13:53:59.885 INFO:teuthology.orchestra.run.smithi116.stdout:(124/133): python3-trustme-0.6.0-4.el8.noarch.r 1.4 MB/s | 27 kB 00:00 2020-11-20T13:53:59.900 INFO:teuthology.orchestra.run.smithi116.stdout:(125/133): python3-waitress-1.2.1-2.el8.1.noarc 13 MB/s | 241 kB 00:00 2020-11-20T13:53:59.906 INFO:teuthology.orchestra.run.smithi116.stdout:(126/133): python3-webob-1.8.5-1.el8.1.noarch.r 12 MB/s | 251 kB 00:00 2020-11-20T13:53:59.916 INFO:teuthology.orchestra.run.smithi116.stdout:(127/133): python3-websocket-client-0.56.0-5.el 3.6 MB/s | 61 kB 00:00 2020-11-20T13:53:59.926 INFO:teuthology.orchestra.run.smithi116.stdout:(128/133): python3-webtest-2.0.33-1.el8.noarch. 4.1 MB/s | 85 kB 00:00 2020-11-20T13:53:59.933 INFO:teuthology.orchestra.run.smithi116.stdout:(129/133): python3-zc-lockfile-2.0-2.el8.noarch 1.4 MB/s | 23 kB 00:00 2020-11-20T13:53:59.947 INFO:teuthology.orchestra.run.smithi116.stdout:(130/133): xmlstarlet-1.6.1-11.el8.x86_64.rpm 3.4 MB/s | 69 kB 00:00 2020-11-20T13:54:00.335 INFO:teuthology.orchestra.run.smithi116.stdout:(131/133): librbd1-15.2.5-588.ga7b62340.el8.x86 5.0 MB/s | 1.9 MB 00:00 2020-11-20T13:54:00.691 INFO:teuthology.orchestra.run.smithi116.stdout:(132/133): librados2-15.2.5-588.ga7b62340.el8.x 4.7 MB/s | 3.6 MB 00:00 2020-11-20T13:54:01.410 INFO:teuthology.orchestra.run.smithi116.stdout:(133/133): ceph-test-15.2.5-588.ga7b62340.el8.x 9.4 MB/s | 42 MB 00:04 2020-11-20T13:54:01.415 INFO:teuthology.orchestra.run.smithi116.stdout:-------------------------------------------------------------------------------- 2020-11-20T13:54:01.416 INFO:teuthology.orchestra.run.smithi116.stdout:Total 26 MB/s | 185 MB 00:07 2020-11-20T13:54:01.478 INFO:teuthology.orchestra.run.smithi116.stdout:Running transaction check 2020-11-20T13:54:01.642 INFO:teuthology.orchestra.run.smithi116.stdout:Transaction check succeeded. 2020-11-20T13:54:01.643 INFO:teuthology.orchestra.run.smithi116.stdout:Running transaction test 2020-11-20T13:54:02.057 INFO:teuthology.orchestra.run.smithi040.stdout:(15/133): ceph-radosgw-15.2.5-588.ga7b62340.el8 2.9 MB/s | 10 MB 00:03 2020-11-20T13:54:02.101 INFO:teuthology.orchestra.run.smithi040.stdout:(16/133): cephadm-15.2.5-588.ga7b62340.el8.x86_ 1.3 MB/s | 57 kB 00:00 2020-11-20T13:54:02.142 INFO:teuthology.orchestra.run.smithi040.stdout:(17/133): libcephfs-devel-15.2.5-588.ga7b62340. 528 kB/s | 22 kB 00:00 2020-11-20T13:54:02.344 INFO:teuthology.orchestra.run.smithi040.stdout:(18/133): libcephfs2-15.2.5-588.ga7b62340.el8.x 2.5 MB/s | 511 kB 00:00 2020-11-20T13:54:02.400 INFO:teuthology.orchestra.run.smithi040.stdout:(19/133): librados-devel-15.2.5-588.ga7b62340.e 2.5 MB/s | 141 kB 00:00 2020-11-20T13:54:02.572 INFO:teuthology.orchestra.run.smithi040.stdout:(20/133): libradosstriper1-15.2.5-588.ga7b62340 2.2 MB/s | 381 kB 00:00 2020-11-20T13:54:03.026 INFO:teuthology.orchestra.run.smithi116.stdout:Transaction test succeeded. 2020-11-20T13:54:03.026 INFO:teuthology.orchestra.run.smithi116.stdout:Running transaction 2020-11-20T13:54:03.373 INFO:teuthology.orchestra.run.smithi040.stdout:(21/133): ceph-test-15.2.5-588.ga7b62340.el8.x8 12 MB/s | 42 MB 00:03 2020-11-20T13:54:03.442 INFO:teuthology.orchestra.run.smithi040.stdout:(22/133): python3-ceph-argparse-15.2.5-588.ga7b 575 kB/s | 40 kB 00:00 2020-11-20T13:54:03.525 INFO:teuthology.orchestra.run.smithi040.stdout:(23/133): python3-ceph-common-15.2.5-588.ga7b62 888 kB/s | 73 kB 00:00 2020-11-20T13:54:03.600 INFO:teuthology.orchestra.run.smithi040.stdout:(24/133): python3-cephfs-15.2.5-588.ga7b62340.e 2.1 MB/s | 163 kB 00:00 2020-11-20T13:54:03.755 INFO:teuthology.orchestra.run.smithi040.stdout:(25/133): python3-rados-15.2.5-588.ga7b62340.el 2.2 MB/s | 351 kB 00:00 2020-11-20T13:54:03.838 INFO:teuthology.orchestra.run.smithi040.stdout:(26/133): python3-rbd-15.2.5-588.ga7b62340.el8. 3.9 MB/s | 334 kB 00:00 2020-11-20T13:54:03.897 INFO:teuthology.orchestra.run.smithi040.stdout:(27/133): python3-rgw-15.2.5-588.ga7b62340.el8. 1.9 MB/s | 113 kB 00:00 2020-11-20T13:54:03.954 INFO:teuthology.orchestra.run.smithi040.stdout:(28/133): rbd-fuse-15.2.5-588.ga7b62340.el8.x86 1.2 MB/s | 72 kB 00:00 2020-11-20T13:54:04.127 INFO:teuthology.orchestra.run.smithi040.stdout:(29/133): librgw2-15.2.5-588.ga7b62340.el8.x86_ 2.1 MB/s | 3.2 MB 00:01 2020-11-20T13:54:04.416 INFO:teuthology.orchestra.run.smithi040.stdout:(30/133): ceph-common-15.2.5-588.ga7b62340.el8. 2.1 MB/s | 21 MB 00:09 2020-11-20T13:54:04.447 INFO:teuthology.orchestra.run.smithi040.stdout:(31/133): rbd-nbd-15.2.5-588.ga7b62340.el8.x86_ 443 kB/s | 141 kB 00:00 2020-11-20T13:54:04.457 INFO:teuthology.orchestra.run.smithi040.stdout:(32/133): rbd-mirror-15.2.5-588.ga7b62340.el8.x 5.4 MB/s | 2.7 MB 00:00 2020-11-20T13:54:04.470 INFO:teuthology.orchestra.run.smithi040.stdout:(33/133): ceph-grafana-dashboards-15.2.5-588.ga 452 kB/s | 24 kB 00:00 2020-11-20T13:54:04.503 INFO:teuthology.orchestra.run.smithi040.stdout:(34/133): ceph-mgr-cephadm-15.2.5-588.ga7b62340 1.1 MB/s | 65 kB 00:00 2020-11-20T13:54:04.666 INFO:teuthology.orchestra.run.smithi040.stdout:(35/133): ceph-mgr-modules-core-15.2.5-588.ga7b 1.1 MB/s | 176 kB 00:00 2020-11-20T13:54:04.745 INFO:teuthology.orchestra.run.smithi040.stdout:(36/133): ceph-mgr-rook-15.2.5-588.ga7b62340.el 345 kB/s | 27 kB 00:00 2020-11-20T13:54:04.813 INFO:teuthology.orchestra.run.smithi040.stdout:(37/133): ceph-prometheus-alerts-15.2.5-588.ga7 128 kB/s | 8.7 kB 00:00 2020-11-20T13:54:04.854 INFO:teuthology.orchestra.run.smithi040.stdout:(38/133): automake-1.16.1-6.el8.noarch.rpm 17 MB/s | 713 kB 00:00 2020-11-20T13:54:04.863 INFO:teuthology.orchestra.run.smithi040.stdout:(39/133): bison-3.0.4-10.el8.x86_64.rpm 76 MB/s | 688 kB 00:00 2020-11-20T13:54:04.868 INFO:teuthology.orchestra.run.smithi040.stdout:(40/133): flex-2.6.1-9.el8.x86_64.rpm 69 MB/s | 320 kB 00:00 2020-11-20T13:54:04.871 INFO:teuthology.orchestra.run.smithi040.stdout:(41/133): jq-1.5-12.el8.x86_64.rpm 58 MB/s | 161 kB 00:00 2020-11-20T13:54:04.876 INFO:teuthology.orchestra.run.smithi040.stdout:(42/133): librdkafka-0.11.4-1.el8.x86_64.rpm 69 MB/s | 353 kB 00:00 2020-11-20T13:54:04.885 INFO:teuthology.orchestra.run.smithi040.stdout:(43/133): libtool-2.4.6-25.el8.x86_64.rpm 77 MB/s | 709 kB 00:00 2020-11-20T13:54:04.889 INFO:teuthology.orchestra.run.smithi040.stdout:(44/133): lttng-ust-2.8.1-11.el8.x86_64.rpm 66 MB/s | 259 kB 00:00 2020-11-20T13:54:04.893 INFO:teuthology.orchestra.run.smithi040.stdout:(45/133): oniguruma-6.8.2-1.el8.x86_64.rpm 61 MB/s | 188 kB 00:00 2020-11-20T13:54:04.944 INFO:teuthology.orchestra.run.smithi040.stdout:(46/133): openblas-0.3.3-2.el8.x86_64.rpm 84 MB/s | 4.3 MB 00:00 2020-11-20T13:54:04.965 INFO:teuthology.orchestra.run.smithi040.stdout:(47/133): ceph-mgr-dashboard-15.2.5-588.ga7b623 7.0 MB/s | 3.5 MB 00:00 2020-11-20T13:54:05.006 INFO:teuthology.orchestra.run.smithi040.stdout:(48/133): openblas-threads-0.3.3-2.el8.x86_64.r 71 MB/s | 4.4 MB 00:00 2020-11-20T13:54:05.010 INFO:teuthology.orchestra.run.smithi040.stdout:(49/133): python3-html5lib-0.999999999-6.el8.no 63 MB/s | 214 kB 00:00 2020-11-20T13:54:05.018 INFO:teuthology.orchestra.run.smithi040.stdout:(50/133): python3-jinja2-2.10.1-2.el8_0.noarch. 68 MB/s | 538 kB 00:00 2020-11-20T13:54:05.019 INFO:teuthology.orchestra.run.smithi040.stdout:(51/133): python3-jsonpatch-1.21-2.el8.noarch.r 18 MB/s | 27 kB 00:00 2020-11-20T13:54:05.021 INFO:teuthology.orchestra.run.smithi040.stdout:(52/133): python3-jsonpointer-1.10-11.el8.noarc 15 MB/s | 20 kB 00:00 2020-11-20T13:54:05.043 INFO:teuthology.orchestra.run.smithi040.stdout:(53/133): python3-lxml-4.2.3-1.el8.x86_64.rpm 71 MB/s | 1.5 MB 00:00 2020-11-20T13:54:05.046 INFO:teuthology.orchestra.run.smithi040.stdout:(54/133): python3-mako-1.0.6-13.el8.noarch.rpm 40 MB/s | 157 kB 00:00 2020-11-20T13:54:05.049 INFO:teuthology.orchestra.run.smithi040.stdout:(55/133): python3-markupsafe-0.23-19.el8.x86_64 20 MB/s | 39 kB 00:00 2020-11-20T13:54:05.102 INFO:teuthology.orchestra.run.smithi040.stdout:(56/133): python3-numpy-1.14.3-9.el8.x86_64.rpm 70 MB/s | 3.7 MB 00:00 2020-11-20T13:54:05.108 INFO:teuthology.orchestra.run.smithi040.stdout:(57/133): python3-numpy-f2py-1.14.3-9.el8.x86_6 42 MB/s | 225 kB 00:00 2020-11-20T13:54:05.110 INFO:teuthology.orchestra.run.smithi040.stdout:(58/133): python3-prettytable-0.7.2-14.el8.noar 18 MB/s | 44 kB 00:00 2020-11-20T13:54:05.115 INFO:teuthology.orchestra.run.smithi040.stdout:(59/133): python3-pyasn1-0.3.7-6.el8.noarch.rpm 31 MB/s | 126 kB 00:00 2020-11-20T13:54:05.118 INFO:teuthology.orchestra.run.smithi040.stdout:(60/133): python3-pyasn1-modules-0.3.7-6.el8.no 31 MB/s | 110 kB 00:00 2020-11-20T13:54:05.121 INFO:teuthology.orchestra.run.smithi040.stdout:(61/133): python3-pytz-2017.2-9.el8.noarch.rpm 18 MB/s | 54 kB 00:00 2020-11-20T13:54:05.136 INFO:teuthology.orchestra.run.smithi040.stdout:(62/133): python3-babel-2.5.1-5.el8.noarch.rpm 28 MB/s | 4.8 MB 00:00 2020-11-20T13:54:05.141 INFO:teuthology.orchestra.run.smithi040.stdout:(63/133): python3-webencodings-0.5.1-6.el8.noar 4.6 MB/s | 27 kB 00:00 2020-11-20T13:54:05.165 INFO:teuthology.orchestra.run.smithi040.stdout:(64/133): python3-werkzeug-0.12.2-4.el8.noarch. 20 MB/s | 457 kB 00:00 2020-11-20T13:54:05.182 INFO:teuthology.orchestra.run.smithi040.stdout:(65/133): socat-1.7.3.2-6.el8.x86_64.rpm 17 MB/s | 298 kB 00:00 2020-11-20T13:54:05.190 INFO:teuthology.orchestra.run.smithi040.stdout:(66/133): elfutils-libelf-devel-0.176-5.el8.x86 6.6 MB/s | 54 kB 00:00 2020-11-20T13:54:05.199 INFO:teuthology.orchestra.run.smithi040.stdout:(67/133): fuse-2.9.7-12.el8.x86_64.rpm 9.0 MB/s | 82 kB 00:00 2020-11-20T13:54:05.207 INFO:teuthology.orchestra.run.smithi040.stdout:(68/133): fuse-common-3.2.1-12.el8.x86_64.rpm 2.5 MB/s | 21 kB 00:00 2020-11-20T13:54:05.216 INFO:teuthology.orchestra.run.smithi040.stdout:(69/133): keyutils-libs-devel-1.5.10-6.el8.x86_ 5.4 MB/s | 48 kB 00:00 2020-11-20T13:54:05.248 INFO:teuthology.orchestra.run.smithi040.stdout:(70/133): krb5-devel-1.17-9.el8.x86_64.rpm 17 MB/s | 548 kB 00:00 2020-11-20T13:54:05.258 INFO:teuthology.orchestra.run.smithi040.stdout:(71/133): libaio-devel-0.3.112-1.el8.x86_64.rpm 1.9 MB/s | 19 kB 00:00 2020-11-20T13:54:05.269 INFO:teuthology.orchestra.run.smithi040.stdout:(72/133): libcom_err-devel-1.44.6-3.el8.x86_64. 3.6 MB/s | 38 kB 00:00 2020-11-20T13:54:05.376 INFO:teuthology.orchestra.run.smithi040.stdout:(73/133): python3-scipy-1.0.0-20.module_el8.1.0 57 MB/s | 15 MB 00:00 2020-11-20T13:54:05.379 INFO:teuthology.orchestra.run.smithi040.stdout:(74/133): libquadmath-8.3.1-4.5.el8.x86_64.rpm 58 MB/s | 169 kB 00:00 2020-11-20T13:54:05.380 INFO:teuthology.orchestra.run.smithi040.stdout:(75/133): librabbitmq-0.9.0-1.el8.x86_64.rpm 34 MB/s | 47 kB 00:00 2020-11-20T13:54:05.382 INFO:teuthology.orchestra.run.smithi040.stdout:(76/133): libgfortran-8.3.1-4.5.el8.x86_64.rpm 5.5 MB/s | 638 kB 00:00 2020-11-20T13:54:05.384 INFO:teuthology.orchestra.run.smithi040.stdout:(77/133): libselinux-devel-2.9-2.1.el8.x86_64.r 51 MB/s | 199 kB 00:00 2020-11-20T13:54:05.385 INFO:teuthology.orchestra.run.smithi040.stdout:(78/133): libsepol-devel-2.9-1.el8.x86_64.rpm 34 MB/s | 86 kB 00:00 2020-11-20T13:54:05.386 INFO:teuthology.orchestra.run.smithi040.stdout:(79/133): libuuid-devel-2.32.1-17.el8.x86_64.rp 52 MB/s | 95 kB 00:00 2020-11-20T13:54:05.399 INFO:teuthology.orchestra.run.smithi040.stdout:(80/133): libverto-devel-0.3.0-5.el8.x86_64.rpm 1.3 MB/s | 18 kB 00:00 2020-11-20T13:54:05.401 INFO:teuthology.orchestra.run.smithi040.stdout:(81/133): libxslt-1.1.32-3.el8.x86_64.rpm 17 MB/s | 249 kB 00:00 2020-11-20T13:54:05.412 INFO:teuthology.orchestra.run.smithi040.stdout:(82/133): pcre2-devel-10.32-1.el8.x86_64.rpm 54 MB/s | 605 kB 00:00 2020-11-20T13:54:05.418 INFO:teuthology.orchestra.run.smithi040.stdout:(83/133): pcre2-utf16-10.32-1.el8.x86_64.rpm 38 MB/s | 228 kB 00:00 2020-11-20T13:54:05.424 INFO:teuthology.orchestra.run.smithi040.stdout:(84/133): pcre2-utf32-10.32-1.el8.x86_64.rpm 38 MB/s | 220 kB 00:00 2020-11-20T13:54:05.431 INFO:teuthology.orchestra.run.smithi040.stdout:(85/133): python3-chardet-3.0.4-7.el8.noarch.rp 32 MB/s | 195 kB 00:00 2020-11-20T13:54:05.438 INFO:teuthology.orchestra.run.smithi040.stdout:(86/133): openssl-devel-1.1.1c-2.el8_1.1.x86_64 59 MB/s | 2.3 MB 00:00 2020-11-20T13:54:05.439 INFO:teuthology.orchestra.run.smithi040.stdout:(87/133): python3-jwt-1.6.1-2.el8.noarch.rpm 5.4 MB/s | 43 kB 00:00 2020-11-20T13:54:05.440 INFO:teuthology.orchestra.run.smithi040.stdout:(88/133): python3-pysocks-1.6.8-3.el8.noarch.rp 18 MB/s | 34 kB 00:00 2020-11-20T13:54:05.441 INFO:teuthology.orchestra.run.smithi040.stdout:(89/133): python3-oauthlib-2.1.0-1.el8.noarch.r 50 MB/s | 155 kB 00:00 2020-11-20T13:54:05.443 INFO:teuthology.orchestra.run.smithi040.stdout:(90/133): python3-requests-2.20.0-2.1.el8_1.noa 55 MB/s | 123 kB 00:00 2020-11-20T13:54:05.443 INFO:teuthology.orchestra.run.smithi040.stdout:(91/133): python3-requests-oauthlib-1.0.0-1.el8 20 MB/s | 43 kB 00:00 2020-11-20T13:54:05.446 INFO:teuthology.orchestra.run.smithi040.stdout:(92/133): python3-urllib3-1.24.2-2.el8.noarch.r 57 MB/s | 176 kB 00:00 2020-11-20T13:54:05.447 INFO:teuthology.orchestra.run.smithi040.stdout:(93/133): xfsprogs-devel-5.0.0-1.el8.x86_64.rpm 21 MB/s | 69 kB 00:00 2020-11-20T13:54:05.448 INFO:teuthology.orchestra.run.smithi040.stdout:(94/133): zlib-devel-1.2.11-10.el8.x86_64.rpm 38 MB/s | 56 kB 00:00 2020-11-20T13:54:05.930 INFO:teuthology.orchestra.run.smithi040.stdout:(95/133): gperftools-libs-2.7-6.el8.x86_64.rpm 654 kB/s | 315 kB 00:00 2020-11-20T13:54:05.930 INFO:teuthology.orchestra.run.smithi040.stdout:(96/133): fmt-6.2.1-1.el8.x86_64.rpm 240 kB/s | 116 kB 00:00 2020-11-20T13:54:05.947 INFO:teuthology.orchestra.run.smithi040.stdout:(97/133): leveldb-1.22-1.el8.x86_64.rpm 10 MB/s | 181 kB 00:00 2020-11-20T13:54:05.950 INFO:teuthology.orchestra.run.smithi040.stdout:(98/133): liboath-2.6.2-3.el8.x86_64.rpm 2.9 MB/s | 59 kB 00:00 2020-11-20T13:54:05.964 INFO:teuthology.orchestra.run.smithi040.stdout:(99/133): libunwind-1.3.1-3.el8.x86_64.rpm 4.3 MB/s | 75 kB 00:00 2020-11-20T13:54:05.970 INFO:teuthology.orchestra.run.smithi040.stdout:(100/133): python3-apipkg-1.5-6.el8.noarch.rpm 1.1 MB/s | 21 kB 00:00 2020-11-20T13:54:05.981 INFO:teuthology.orchestra.run.smithi040.stdout:(101/133): python3-bcrypt-3.1.6-2.el8.1.x86_64. 2.6 MB/s | 44 kB 00:00 2020-11-20T13:54:05.997 INFO:teuthology.orchestra.run.smithi040.stdout:(102/133): python3-cachetools-3.1.1-4.el8.noarc 2.0 MB/s | 33 kB 00:00 2020-11-20T13:54:06.010 INFO:teuthology.orchestra.run.smithi040.stdout:(103/133): python3-beautifulsoup4-4.6.3-2.el8.1 4.6 MB/s | 185 kB 00:00 2020-11-20T13:54:06.014 INFO:teuthology.orchestra.run.smithi040.stdout:(104/133): python3-certifi-2018.10.15-7.el8.noa 1.0 MB/s | 16 kB 00:00 2020-11-20T13:54:06.031 INFO:teuthology.orchestra.run.smithi040.stdout:(105/133): python3-cheroot-8.2.1-1.el8.noarch.r 6.7 MB/s | 145 kB 00:00 2020-11-20T13:54:06.047 INFO:teuthology.orchestra.run.smithi040.stdout:(106/133): python3-cherrypy-18.4.0-1.el8.noarch 11 MB/s | 384 kB 00:00 2020-11-20T13:54:06.051 INFO:teuthology.orchestra.run.smithi040.stdout:(107/133): python3-cssselect-0.9.2-10.el8.noarc 2.0 MB/s | 40 kB 00:00 2020-11-20T13:54:06.066 INFO:teuthology.orchestra.run.smithi040.stdout:(108/133): python3-execnet-1.7.1-1.el8.noarch.r 19 MB/s | 355 kB 00:00 2020-11-20T13:54:06.071 INFO:teuthology.orchestra.run.smithi040.stdout:(109/133): python3-google-auth-1.1.1-10.el8.noa 3.9 MB/s | 81 kB 00:00 2020-11-20T13:54:06.082 INFO:teuthology.orchestra.run.smithi040.stdout:(110/133): python3-jaraco-6.2-6.el8.noarch.rpm 683 kB/s | 11 kB 00:00 2020-11-20T13:54:06.091 INFO:teuthology.orchestra.run.smithi040.stdout:(111/133): python3-jaraco-functools-2.0-4.el8.n 884 kB/s | 18 kB 00:00 2020-11-20T13:54:06.111 INFO:teuthology.orchestra.run.smithi040.stdout:(112/133): python3-logutils-0.3.5-11.el8.noarch 2.4 MB/s | 49 kB 00:00 2020-11-20T13:54:06.131 INFO:teuthology.orchestra.run.smithi040.stdout:(113/133): python3-more-itertools-7.2.0-3.el8.n 2.9 MB/s | 59 kB 00:00 2020-11-20T13:54:06.149 INFO:teuthology.orchestra.run.smithi040.stdout:(114/133): python3-kubernetes-11.0.0-2.el8.noar 17 MB/s | 1.2 MB 00:00 2020-11-20T13:54:06.153 INFO:teuthology.orchestra.run.smithi040.stdout:(115/133): python3-pecan-1.3.2-9.el8.noarch.rpm 13 MB/s | 283 kB 00:00 2020-11-20T13:54:06.167 INFO:teuthology.orchestra.run.smithi040.stdout:(116/133): python3-portend-2.6-1.el8.noarch.rpm 898 kB/s | 16 kB 00:00 2020-11-20T13:54:06.173 INFO:teuthology.orchestra.run.smithi040.stdout:(117/133): python3-remoto-1.1.4-4.el8.noarch.rp 2.3 MB/s | 48 kB 00:00 2020-11-20T13:54:06.183 INFO:teuthology.orchestra.run.smithi040.stdout:(118/133): python3-repoze-lru-0.7-6.el8.noarch. 2.0 MB/s | 34 kB 00:00 2020-11-20T13:54:06.194 INFO:teuthology.orchestra.run.smithi040.stdout:(119/133): python3-routes-2.4.1-12.el8.noarch.r 9.2 MB/s | 196 kB 00:00 2020-11-20T13:54:06.200 INFO:teuthology.orchestra.run.smithi040.stdout:(120/133): python3-rsa-3.4.2-15.el8.noarch.rpm 4.3 MB/s | 73 kB 00:00 2020-11-20T13:54:06.214 INFO:teuthology.orchestra.run.smithi040.stdout:(121/133): python3-simplegeneric-0.8.1-17.el8.n 947 kB/s | 19 kB 00:00 2020-11-20T13:54:06.217 INFO:teuthology.orchestra.run.smithi040.stdout:(122/133): python3-singledispatch-3.4.0.3-18.el 1.4 MB/s | 24 kB 00:00 2020-11-20T13:54:06.234 INFO:teuthology.orchestra.run.smithi040.stdout:(123/133): python3-trustme-0.6.0-4.el8.noarch.r 1.6 MB/s | 27 kB 00:00 2020-11-20T13:54:06.234 INFO:teuthology.orchestra.run.smithi040.stdout:(124/133): python3-tempora-1.14.1-5.el8.noarch. 1.5 MB/s | 29 kB 00:00 2020-11-20T13:54:06.252 INFO:teuthology.orchestra.run.smithi040.stdout:(125/133): python3-waitress-1.2.1-2.el8.1.noarc 13 MB/s | 241 kB 00:00 2020-11-20T13:54:06.256 INFO:teuthology.orchestra.run.smithi040.stdout:(126/133): python3-webob-1.8.5-1.el8.1.noarch.r 12 MB/s | 251 kB 00:00 2020-11-20T13:54:06.269 INFO:teuthology.orchestra.run.smithi040.stdout:(127/133): python3-websocket-client-0.56.0-5.el 3.6 MB/s | 61 kB 00:00 2020-11-20T13:54:06.276 INFO:teuthology.orchestra.run.smithi040.stdout:(128/133): python3-webtest-2.0.33-1.el8.noarch. 4.1 MB/s | 85 kB 00:00 2020-11-20T13:54:06.286 INFO:teuthology.orchestra.run.smithi040.stdout:(129/133): python3-zc-lockfile-2.0-2.el8.noarch 1.4 MB/s | 23 kB 00:00 2020-11-20T13:54:06.296 INFO:teuthology.orchestra.run.smithi040.stdout:(130/133): xmlstarlet-1.6.1-11.el8.x86_64.rpm 3.4 MB/s | 69 kB 00:00 2020-11-20T13:54:07.067 INFO:teuthology.orchestra.run.smithi040.stdout:(131/133): librados2-15.2.5-588.ga7b62340.el8.x 4.6 MB/s | 3.6 MB 00:00 2020-11-20T13:54:07.294 INFO:teuthology.orchestra.run.smithi040.stdout:(132/133): librbd1-15.2.5-588.ga7b62340.el8.x86 1.9 MB/s | 1.9 MB 00:00 2020-11-20T13:54:07.411 INFO:teuthology.orchestra.run.smithi040.stdout:(133/133): ceph-mgr-diskprediction-local-15.2.5 2.9 MB/s | 8.6 MB 00:02 2020-11-20T13:54:07.416 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2020-11-20T13:54:07.416 INFO:teuthology.orchestra.run.smithi040.stdout:Total 14 MB/s | 185 MB 00:12 2020-11-20T13:54:07.485 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2020-11-20T13:54:07.666 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2020-11-20T13:54:07.667 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2020-11-20T13:54:08.280 INFO:teuthology.orchestra.run.smithi116.stdout: Preparing : 1/1 2020-11-20T13:54:08.526 INFO:teuthology.orchestra.run.smithi116.stdout: Downgrading : openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 1/140 2020-11-20T13:54:08.599 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 1/140 2020-11-20T13:54:08.795 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : fmt-6.2.1-1.el8.x86_64 2/140 2020-11-20T13:54:08.981 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : liboath-2.6.2-3.el8.x86_64 3/140 2020-11-20T13:54:09.116 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2020-11-20T13:54:09.117 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2020-11-20T13:54:09.198 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : leveldb-1.22-1.el8.x86_64 4/140 2020-11-20T13:54:09.331 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 5/140 2020-11-20T13:54:09.426 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 5/140 2020-11-20T13:54:09.598 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : librabbitmq-0.9.0-1.el8.x86_64 6/140 2020-11-20T13:54:09.789 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 7/140 2020-11-20T13:54:10.187 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 7/140 2020-11-20T13:54:10.389 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : librados2-2:15.2.5-588.ga7b62340.el8.x86_64 8/140 2020-11-20T13:54:10.608 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: librados2-2:15.2.5-588.ga7b62340.el8.x86_64 8/140 2020-11-20T13:54:10.872 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : librbd1-2:15.2.5-588.ga7b62340.el8.x86_64 9/140 2020-11-20T13:54:10.970 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: librbd1-2:15.2.5-588.ga7b62340.el8.x86_64 9/140 2020-11-20T13:54:11.173 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libcephfs2-2:15.2.5-588.ga7b62340.el8.x86_64 10/140 2020-11-20T13:54:11.262 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: libcephfs2-2:15.2.5-588.ga7b62340.el8.x86_64 10/140 2020-11-20T13:54:11.742 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-rados-2:15.2.5-588.ga7b62340.el8.x86_64 11/140 2020-11-20T13:54:11.914 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : librgw2-2:15.2.5-588.ga7b62340.el8.x86_64 12/140 2020-11-20T13:54:12.076 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: librgw2-2:15.2.5-588.ga7b62340.el8.x86_64 12/140 2020-11-20T13:54:12.344 INFO:teuthology.orchestra.run.smithi116.stdout: Downgrading : krb5-libs-1.17-9.el8.x86_64 13/140 2020-11-20T13:54:12.509 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 14/140 2020-11-20T13:54:12.698 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 15/140 2020-11-20T13:54:12.914 INFO:teuthology.orchestra.run.smithi116.stdout: Downgrading : libkadm5-1.17-9.el8.x86_64 16/140 2020-11-20T13:54:13.080 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libradosstriper1-2:15.2.5-588.ga7b62340.el8.x86_ 17/140 2020-11-20T13:54:13.184 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: libradosstriper1-2:15.2.5-588.ga7b62340.el8.x86_ 17/140 2020-11-20T13:54:13.420 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 18/140 2020-11-20T13:54:13.557 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : zlib-devel-1.2.11-10.el8.x86_64 19/140 2020-11-20T13:54:13.713 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 20/140 2020-11-20T13:54:13.916 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libxslt-1.1.32-3.el8.x86_64 21/140 2020-11-20T13:54:14.097 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libquadmath-8.3.1-4.5.el8.x86_64 22/140 2020-11-20T13:54:14.233 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: libquadmath-8.3.1-4.5.el8.x86_64 22/140 2020-11-20T13:54:14.405 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libgfortran-8.3.1-4.5.el8.x86_64 23/140 2020-11-20T13:54:14.600 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2020-11-20T13:54:14.845 INFO:teuthology.orchestra.run.smithi040.stdout: Downgrading : openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 1/140 2020-11-20T13:54:14.920 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 1/140 2020-11-20T13:54:14.924 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: libgfortran-8.3.1-4.5.el8.x86_64 23/140 2020-11-20T13:54:15.080 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : openblas-0.3.3-2.el8.x86_64 24/140 2020-11-20T13:54:15.084 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : fmt-6.2.1-1.el8.x86_64 2/140 2020-11-20T13:54:15.255 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : liboath-2.6.2-3.el8.x86_64 3/140 2020-11-20T13:54:15.422 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : leveldb-1.22-1.el8.x86_64 4/140 2020-11-20T13:54:15.553 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 5/140 2020-11-20T13:54:15.606 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: openblas-0.3.3-2.el8.x86_64 24/140 2020-11-20T13:54:15.611 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 5/140 2020-11-20T13:54:15.763 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : openblas-threads-0.3.3-2.el8.x86_64 25/140 2020-11-20T13:54:15.821 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librabbitmq-0.9.0-1.el8.x86_64 6/140 2020-11-20T13:54:15.995 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 7/140 2020-11-20T13:54:16.431 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 7/140 2020-11-20T13:54:16.447 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: openblas-threads-0.3.3-2.el8.x86_64 25/140 2020-11-20T13:54:16.595 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : librados2-2:15.2.5-588.ga7b62340.el8.x86_64 8/140 2020-11-20T13:54:16.815 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 26/140 2020-11-20T13:54:16.843 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librados2-2:15.2.5-588.ga7b62340.el8.x86_64 8/140 2020-11-20T13:54:17.027 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 27/140 2020-11-20T13:54:17.078 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : librbd1-2:15.2.5-588.ga7b62340.el8.x86_64 9/140 2020-11-20T13:54:17.179 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 28/140 2020-11-20T13:54:17.185 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librbd1-2:15.2.5-588.ga7b62340.el8.x86_64 9/140 2020-11-20T13:54:17.362 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephfs2-2:15.2.5-588.ga7b62340.el8.x86_64 10/140 2020-11-20T13:54:17.369 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 29/140 2020-11-20T13:54:17.459 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libcephfs2-2:15.2.5-588.ga7b62340.el8.x86_64 10/140 2020-11-20T13:54:17.559 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-ceph-common-2:15.2.5-588.ga7b62340.el8.x 30/140 2020-11-20T13:54:17.722 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-ceph-argparse-2:15.2.5-588.ga7b62340.el8 31/140 2020-11-20T13:54:17.900 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rados-2:15.2.5-588.ga7b62340.el8.x86_64 11/140 2020-11-20T13:54:17.916 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-cephfs-2:15.2.5-588.ga7b62340.el8.x86_64 32/140 2020-11-20T13:54:18.052 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librgw2-2:15.2.5-588.ga7b62340.el8.x86_64 12/140 2020-11-20T13:54:18.219 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librgw2-2:15.2.5-588.ga7b62340.el8.x86_64 12/140 2020-11-20T13:54:18.458 INFO:teuthology.orchestra.run.smithi040.stdout: Downgrading : krb5-libs-1.17-9.el8.x86_64 13/140 2020-11-20T13:54:18.753 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 14/140 2020-11-20T13:54:18.836 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/140 2020-11-20T13:54:18.963 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 15/140 2020-11-20T13:54:19.146 INFO:teuthology.orchestra.run.smithi040.stdout: Downgrading : libkadm5-1.17-9.el8.x86_64 16/140 2020-11-20T13:54:19.178 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 34/140 2020-11-20T13:54:19.286 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libradosstriper1-2:15.2.5-588.ga7b62340.el8.x86_ 17/140 2020-11-20T13:54:19.410 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libradosstriper1-2:15.2.5-588.ga7b62340.el8.x86_ 17/140 2020-11-20T13:54:19.467 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 35/140 2020-11-20T13:54:19.651 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 18/140 2020-11-20T13:54:19.696 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 36/140 2020-11-20T13:54:19.797 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : zlib-devel-1.2.11-10.el8.x86_64 19/140 2020-11-20T13:54:19.970 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 20/140 2020-11-20T13:54:20.134 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libxslt-1.1.32-3.el8.x86_64 21/140 2020-11-20T13:54:20.319 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libquadmath-8.3.1-4.5.el8.x86_64 22/140 2020-11-20T13:54:20.445 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libquadmath-8.3.1-4.5.el8.x86_64 22/140 2020-11-20T13:54:20.627 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libgfortran-8.3.1-4.5.el8.x86_64 23/140 2020-11-20T13:54:21.177 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libgfortran-8.3.1-4.5.el8.x86_64 23/140 2020-11-20T13:54:21.319 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : openblas-0.3.3-2.el8.x86_64 24/140 2020-11-20T13:54:21.650 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 36/140 2020-11-20T13:54:21.893 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: openblas-0.3.3-2.el8.x86_64 24/140 2020-11-20T13:54:21.956 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-scipy-1.0.0-20.module_el8.1.0+245+c39af4 37/140 2020-11-20T13:54:22.019 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : openblas-threads-0.3.3-2.el8.x86_64 25/140 2020-11-20T13:54:22.208 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : xmlstarlet-1.6.1-11.el8.x86_64 38/140 2020-11-20T13:54:22.465 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 39/140 2020-11-20T13:54:22.735 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/140 2020-11-20T13:54:22.784 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: openblas-threads-0.3.3-2.el8.x86_64 25/140 2020-11-20T13:54:22.958 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-rsa-3.4.2-15.el8.noarch 41/140 2020-11-20T13:54:23.058 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 26/140 2020-11-20T13:54:23.195 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-rgw-2:15.2.5-588.ga7b62340.el8.x86_64 42/140 2020-11-20T13:54:23.251 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 27/140 2020-11-20T13:54:23.376 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-rbd-2:15.2.5-588.ga7b62340.el8.x86_64 43/140 2020-11-20T13:54:23.449 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 28/140 2020-11-20T13:54:23.572 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : librados-devel-2:15.2.5-588.ga7b62340.el8.x86_64 44/140 2020-11-20T13:54:23.636 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 29/140 2020-11-20T13:54:23.728 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : socat-1.7.3.2-6.el8.x86_64 45/140 2020-11-20T13:54:23.906 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 46/140 2020-11-20T13:54:24.161 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 47/140 2020-11-20T13:54:24.174 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-ceph-common-2:15.2.5-588.ga7b62340.el8.x 30/140 2020-11-20T13:54:24.313 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-ceph-argparse-2:15.2.5-588.ga7b62340.el8 31/140 2020-11-20T13:54:24.386 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 48/140 2020-11-20T13:54:24.549 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cephfs-2:15.2.5-588.ga7b62340.el8.x86_64 32/140 2020-11-20T13:54:24.568 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 49/140 2020-11-20T13:54:24.724 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 50/140 2020-11-20T13:54:24.897 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 51/140 2020-11-20T13:54:25.128 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 52/140 2020-11-20T13:54:25.330 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 53/140 2020-11-20T13:54:25.490 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 54/140 2020-11-20T13:54:25.543 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/140 2020-11-20T13:54:25.651 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 55/140 2020-11-20T13:54:25.857 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 56/140 2020-11-20T13:54:25.860 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 34/140 2020-11-20T13:54:26.098 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-cheroot-8.2.1-1.el8.noarch 57/140 2020-11-20T13:54:26.127 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 35/140 2020-11-20T13:54:26.360 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 36/140 2020-11-20T13:54:26.418 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 58/140 2020-11-20T13:54:26.650 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-portend-2.6-1.el8.noarch 59/140 2020-11-20T13:54:26.856 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 60/140 2020-11-20T13:54:27.039 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-cssselect-0.9.2-10.el8.noarch 61/140 2020-11-20T13:54:27.201 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 62/140 2020-11-20T13:54:27.418 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 63/140 2020-11-20T13:54:27.639 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 64/140 2020-11-20T13:54:27.852 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 65/140 2020-11-20T13:54:28.125 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-apipkg-1.5-6.el8.noarch 66/140 2020-11-20T13:54:28.367 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-execnet-1.7.1-1.el8.noarch 67/140 2020-11-20T13:54:28.465 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 36/140 2020-11-20T13:54:28.572 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-remoto-1.1.4-4.el8.noarch 68/140 2020-11-20T13:54:28.746 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-scipy-1.0.0-20.module_el8.1.0+245+c39af4 37/140 2020-11-20T13:54:28.748 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 69/140 2020-11-20T13:54:28.961 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : gperftools-libs-2.7-6.el8.x86_64 70/140 2020-11-20T13:54:28.986 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : xmlstarlet-1.6.1-11.el8.x86_64 38/140 2020-11-20T13:54:29.183 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-pysocks-1.6.8-3.el8.noarch 71/140 2020-11-20T13:54:29.241 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 39/140 2020-11-20T13:54:29.440 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-urllib3-1.24.2-2.el8.noarch 72/140 2020-11-20T13:54:29.495 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/140 2020-11-20T13:54:29.642 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-chardet-3.0.4-7.el8.noarch 73/140 2020-11-20T13:54:29.665 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rsa-3.4.2-15.el8.noarch 41/140 2020-11-20T13:54:29.825 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-requests-2.20.0-2.1.el8_1.noarch 74/140 2020-11-20T13:54:29.878 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rgw-2:15.2.5-588.ga7b62340.el8.x86_64 42/140 2020-11-20T13:54:30.050 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rbd-2:15.2.5-588.ga7b62340.el8.x86_64 43/140 2020-11-20T13:54:30.249 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librados-devel-2:15.2.5-588.ga7b62340.el8.x86_64 44/140 2020-11-20T13:54:30.428 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : socat-1.7.3.2-6.el8.x86_64 45/140 2020-11-20T13:54:30.600 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 46/140 2020-11-20T13:54:30.867 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 47/140 2020-11-20T13:54:30.883 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 75/140 2020-11-20T13:54:31.110 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 48/140 2020-11-20T13:54:31.156 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-kubernetes-11.0.0-2.el8.noarch 76/140 2020-11-20T13:54:31.267 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 49/140 2020-11-20T13:54:31.448 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 50/140 2020-11-20T13:54:31.588 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 51/140 2020-11-20T13:54:31.744 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : pcre2-utf32-10.32-1.el8.x86_64 77/140 2020-11-20T13:54:31.813 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 52/140 2020-11-20T13:54:32.031 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 53/140 2020-11-20T13:54:32.070 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : pcre2-utf16-10.32-1.el8.x86_64 78/140 2020-11-20T13:54:32.179 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 54/140 2020-11-20T13:54:32.255 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : pcre2-devel-10.32-1.el8.x86_64 79/140 2020-11-20T13:54:32.341 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 55/140 2020-11-20T13:54:32.393 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 80/140 2020-11-20T13:54:32.528 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 56/140 2020-11-20T13:54:32.557 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libuuid-devel-2.32.1-17.el8.x86_64 81/140 2020-11-20T13:54:32.731 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cheroot-8.2.1-1.el8.noarch 57/140 2020-11-20T13:54:32.857 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libsepol-devel-2.9-1.el8.x86_64 82/140 2020-11-20T13:54:32.924 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 58/140 2020-11-20T13:54:33.073 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libselinux-devel-2.9-2.1.el8.x86_64 83/140 2020-11-20T13:54:33.136 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-portend-2.6-1.el8.noarch 59/140 2020-11-20T13:54:33.240 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libcom_err-devel-1.44.6-3.el8.x86_64 84/140 2020-11-20T13:54:33.363 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 60/140 2020-11-20T13:54:33.442 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 85/140 2020-11-20T13:54:33.510 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cssselect-0.9.2-10.el8.noarch 61/140 2020-11-20T13:54:33.702 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 62/140 2020-11-20T13:54:33.720 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : krb5-devel-1.17-9.el8.x86_64 86/140 2020-11-20T13:54:33.721 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : fuse-common-3.2.1-12.el8.x86_64 87/140 2020-11-20T13:54:33.721 INFO:teuthology.orchestra.run.smithi116.stdout:warning: /etc/fuse.conf created as /etc/fuse.conf.rpmnew 2020-11-20T13:54:33.721 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T13:54:33.931 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 63/140 2020-11-20T13:54:34.112 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 64/140 2020-11-20T13:54:34.113 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : fuse-2.9.7-12.el8.x86_64 88/140 2020-11-20T13:54:34.334 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 65/140 2020-11-20T13:54:34.367 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-webencodings-0.5.1-6.el8.noarch 89/140 2020-11-20T13:54:34.616 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-html5lib-1:0.999999999-6.el8.noarch 90/140 2020-11-20T13:54:34.975 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-apipkg-1.5-6.el8.noarch 66/140 2020-11-20T13:54:35.012 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 91/140 2020-11-20T13:54:35.210 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-execnet-1.7.1-1.el8.noarch 67/140 2020-11-20T13:54:35.297 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-lxml-4.2.3-1.el8.x86_64 92/140 2020-11-20T13:54:35.370 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-remoto-1.1.4-4.el8.noarch 68/140 2020-11-20T13:54:35.557 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 69/140 2020-11-20T13:54:35.619 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 93/140 2020-11-20T13:54:35.751 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : gperftools-libs-2.7-6.el8.x86_64 70/140 2020-11-20T13:54:35.935 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 94/140 2020-11-20T13:54:35.977 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pysocks-1.6.8-3.el8.noarch 71/140 2020-11-20T13:54:36.077 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 95/140 2020-11-20T13:54:36.226 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-urllib3-1.24.2-2.el8.noarch 72/140 2020-11-20T13:54:36.410 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-chardet-3.0.4-7.el8.noarch 73/140 2020-11-20T13:54:36.593 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-requests-2.20.0-2.1.el8_1.noarch 74/140 2020-11-20T13:54:37.734 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 75/140 2020-11-20T13:54:37.954 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-kubernetes-11.0.0-2.el8.noarch 76/140 2020-11-20T13:54:38.170 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : pcre2-utf32-10.32-1.el8.x86_64 77/140 2020-11-20T13:54:38.490 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : pcre2-utf16-10.32-1.el8.x86_64 78/140 2020-11-20T13:54:38.694 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : pcre2-devel-10.32-1.el8.x86_64 79/140 2020-11-20T13:54:38.874 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 80/140 2020-11-20T13:54:39.074 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libuuid-devel-2.32.1-17.el8.x86_64 81/140 2020-11-20T13:54:39.381 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libsepol-devel-2.9-1.el8.x86_64 82/140 2020-11-20T13:54:39.605 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libselinux-devel-2.9-2.1.el8.x86_64 83/140 2020-11-20T13:54:39.765 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcom_err-devel-1.44.6-3.el8.x86_64 84/140 2020-11-20T13:54:39.781 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-common-2:15.2.5-588.ga7b62340.el8.x86_64 96/140 2020-11-20T13:54:40.022 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 85/140 2020-11-20T13:54:40.027 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-common-2:15.2.5-588.ga7b62340.el8.x86_64 96/140 2020-11-20T13:54:40.252 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : krb5-devel-1.17-9.el8.x86_64 86/140 2020-11-20T13:54:40.253 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : fuse-common-3.2.1-12.el8.x86_64 87/140 2020-11-20T13:54:40.253 INFO:teuthology.orchestra.run.smithi040.stdout:warning: /etc/fuse.conf created as /etc/fuse.conf.rpmnew 2020-11-20T13:54:40.253 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:54:40.572 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : fuse-2.9.7-12.el8.x86_64 88/140 2020-11-20T13:54:40.705 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-common-2:15.2.5-588.ga7b62340.el8.x86_64 96/140 2020-11-20T13:54:40.788 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-webencodings-0.5.1-6.el8.noarch 89/140 2020-11-20T13:54:41.009 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-html5lib-1:0.999999999-6.el8.noarch 90/140 2020-11-20T13:54:41.047 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-base-2:15.2.5-588.ga7b62340.el8.x86_64 97/140 2020-11-20T13:54:41.126 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-base-2:15.2.5-588.ga7b62340.el8.x86_64 97/140 2020-11-20T13:54:41.293 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-selinux-2:15.2.5-588.ga7b62340.el8.x86_64 98/140 2020-11-20T13:54:41.417 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 91/140 2020-11-20T13:54:41.685 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-lxml-4.2.3-1.el8.x86_64 92/140 2020-11-20T13:54:42.012 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 93/140 2020-11-20T13:54:42.317 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 94/140 2020-11-20T13:54:42.475 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 95/140 2020-11-20T13:54:46.388 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-common-2:15.2.5-588.ga7b62340.el8.x86_64 96/140 2020-11-20T13:54:46.624 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-common-2:15.2.5-588.ga7b62340.el8.x86_64 96/140 2020-11-20T13:54:47.343 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-common-2:15.2.5-588.ga7b62340.el8.x86_64 96/140 2020-11-20T13:54:47.632 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-base-2:15.2.5-588.ga7b62340.el8.x86_64 97/140 2020-11-20T13:54:47.703 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-base-2:15.2.5-588.ga7b62340.el8.x86_64 97/140 2020-11-20T13:54:47.848 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-selinux-2:15.2.5-588.ga7b62340.el8.x86_64 98/140 2020-11-20T13:54:58.011 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-selinux-2:15.2.5-588.ga7b62340.el8.x86_64 98/140 2020-11-20T13:54:58.012 INFO:teuthology.orchestra.run.smithi116.stdout:skipping the directory /sys 2020-11-20T13:54:58.012 INFO:teuthology.orchestra.run.smithi116.stdout:skipping the directory /proc 2020-11-20T13:54:58.013 INFO:teuthology.orchestra.run.smithi116.stdout:skipping the directory /dev 2020-11-20T13:54:58.013 INFO:teuthology.orchestra.run.smithi116.stdout:skipping the directory /run 2020-11-20T13:54:58.013 INFO:teuthology.orchestra.run.smithi116.stdout:skipping the directory /mnt 2020-11-20T13:54:58.013 INFO:teuthology.orchestra.run.smithi116.stdout:skipping the directory /var/tmp 2020-11-20T13:54:58.014 INFO:teuthology.orchestra.run.smithi116.stdout:skipping the directory /home 2020-11-20T13:54:58.014 INFO:teuthology.orchestra.run.smithi116.stdout:skipping the directory /tmp 2020-11-20T13:54:58.014 INFO:teuthology.orchestra.run.smithi116.stdout:skipping the directory /dev 2020-11-20T13:54:58.014 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T13:54:58.374 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-mds-2:15.2.5-588.ga7b62340.el8.x86_64 99/140 2020-11-20T13:54:58.769 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-mds-2:15.2.5-588.ga7b62340.el8.x86_64 99/140 2020-11-20T13:54:58.961 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-mon-2:15.2.5-588.ga7b62340.el8.x86_64 100/140 2020-11-20T13:55:00.381 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-mon-2:15.2.5-588.ga7b62340.el8.x86_64 100/140 2020-11-20T13:55:00.561 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-osd-2:15.2.5-588.ga7b62340.el8.x86_64 101/140 2020-11-20T13:55:00.591 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-osd-2:15.2.5-588.ga7b62340.el8.x86_64 101/140 2020-11-20T13:55:00.789 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 102/140 2020-11-20T13:55:01.010 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 103/140 2020-11-20T13:55:01.199 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : oniguruma-6.8.2-1.el8.x86_64 104/140 2020-11-20T13:55:01.261 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: oniguruma-6.8.2-1.el8.x86_64 104/140 2020-11-20T13:55:01.546 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : jq-1.5-12.el8.x86_64 105/140 2020-11-20T13:55:01.774 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : automake-1.16.1-6.el8.noarch 106/140 2020-11-20T13:55:01.913 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-prometheus-alerts-2:15.2.5-588.ga7b62340.el 107/140 2020-11-20T13:55:02.107 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-grafana-dashboards-2:15.2.5-588.ga7b62340.e 108/140 2020-11-20T13:55:03.519 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: cephadm-2:15.2.5-588.ga7b62340.el8.x86_64 109/140 2020-11-20T13:55:03.728 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : cephadm-2:15.2.5-588.ga7b62340.el8.x86_64 109/140 2020-11-20T13:55:03.939 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-mgr-cephadm-2:15.2.5-588.ga7b62340.el8.noar 110/140 2020-11-20T13:55:04.811 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-mgr-cephadm-2:15.2.5-588.ga7b62340.el8.noar 110/140 2020-11-20T13:55:05.176 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-mgr-dashboard-2:15.2.5-588.ga7b62340.el8.no 111/140 2020-11-20T13:55:05.660 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-selinux-2:15.2.5-588.ga7b62340.el8.x86_64 98/140 2020-11-20T13:55:05.660 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /sys 2020-11-20T13:55:05.660 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /proc 2020-11-20T13:55:05.661 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /dev 2020-11-20T13:55:05.661 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /run 2020-11-20T13:55:05.661 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /mnt 2020-11-20T13:55:05.661 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /var/tmp 2020-11-20T13:55:05.662 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /home 2020-11-20T13:55:05.662 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /tmp 2020-11-20T13:55:05.662 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /dev 2020-11-20T13:55:05.662 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:55:06.005 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mds-2:15.2.5-588.ga7b62340.el8.x86_64 99/140 2020-11-20T13:55:06.217 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-mgr-dashboard-2:15.2.5-588.ga7b62340.el8.no 111/140 2020-11-20T13:55:06.421 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mds-2:15.2.5-588.ga7b62340.el8.x86_64 99/140 2020-11-20T13:55:06.439 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-mgr-diskprediction-local-2:15.2.5-588.ga7b6 112/140 2020-11-20T13:55:06.510 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:15.2.5-588.ga7b6 112/140 2020-11-20T13:55:06.596 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mon-2:15.2.5-588.ga7b62340.el8.x86_64 100/140 2020-11-20T13:55:06.826 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-mgr-modules-core-2:15.2.5-588.ga7b62340.el8 113/140 2020-11-20T13:55:07.043 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-mgr-2:15.2.5-588.ga7b62340.el8.x86_64 114/140 2020-11-20T13:55:07.070 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-mgr-2:15.2.5-588.ga7b62340.el8.x86_64 114/140 2020-11-20T13:55:07.236 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-mgr-rook-2:15.2.5-588.ga7b62340.el8.noarch 115/140 2020-11-20T13:55:07.249 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-mgr-rook-2:15.2.5-588.ga7b62340.el8.noarch 115/140 2020-11-20T13:55:07.422 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-2:15.2.5-588.ga7b62340.el8.x86_64 116/140 2020-11-20T13:55:07.589 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libtool-2.4.6-25.el8.x86_64 117/140 2020-11-20T13:55:08.100 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mon-2:15.2.5-588.ga7b62340.el8.x86_64 100/140 2020-11-20T13:55:08.263 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-osd-2:15.2.5-588.ga7b62340.el8.x86_64 101/140 2020-11-20T13:55:08.294 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-osd-2:15.2.5-588.ga7b62340.el8.x86_64 101/140 2020-11-20T13:55:08.453 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 102/140 2020-11-20T13:55:08.708 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 103/140 2020-11-20T13:55:08.837 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : oniguruma-6.8.2-1.el8.x86_64 104/140 2020-11-20T13:55:08.920 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: oniguruma-6.8.2-1.el8.x86_64 104/140 2020-11-20T13:55:09.199 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : jq-1.5-12.el8.x86_64 105/140 2020-11-20T13:55:09.455 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : automake-1.16.1-6.el8.noarch 106/140 2020-11-20T13:55:09.595 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-prometheus-alerts-2:15.2.5-588.ga7b62340.el 107/140 2020-11-20T13:55:09.771 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-grafana-dashboards-2:15.2.5-588.ga7b62340.e 108/140 2020-11-20T13:55:11.314 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: cephadm-2:15.2.5-588.ga7b62340.el8.x86_64 109/140 2020-11-20T13:55:11.482 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 117/140 2020-11-20T13:55:11.514 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : cephadm-2:15.2.5-588.ga7b62340.el8.x86_64 109/140 2020-11-20T13:55:11.703 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-cephadm-2:15.2.5-588.ga7b62340.el8.noar 110/140 2020-11-20T13:55:12.635 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-cephadm-2:15.2.5-588.ga7b62340.el8.noar 110/140 2020-11-20T13:55:12.683 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-test-2:15.2.5-588.ga7b62340.el8.x86_64 118/140 2020-11-20T13:55:12.897 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-radosgw-2:15.2.5-588.ga7b62340.el8.x86_64 119/140 2020-11-20T13:55:13.231 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-radosgw-2:15.2.5-588.ga7b62340.el8.x86_64 119/140 2020-11-20T13:55:13.329 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-dashboard-2:15.2.5-588.ga7b62340.el8.no 111/140 2020-11-20T13:55:13.422 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : rbd-mirror-2:15.2.5-588.ga7b62340.el8.x86_64 120/140 2020-11-20T13:55:13.491 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: rbd-mirror-2:15.2.5-588.ga7b62340.el8.x86_64 120/140 2020-11-20T13:55:14.397 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-fuse-2:15.2.5-588.ga7b62340.el8.x86_64 121/140 2020-11-20T13:55:14.435 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-dashboard-2:15.2.5-588.ga7b62340.el8.no 111/140 2020-11-20T13:55:14.670 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-diskprediction-local-2:15.2.5-588.ga7b6 112/140 2020-11-20T13:55:14.674 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : openssl-devel-1:1.1.1c-2.el8_1.1.x86_64 122/140 2020-11-20T13:55:14.746 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:15.2.5-588.ga7b6 112/140 2020-11-20T13:55:14.857 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : xfsprogs-devel-5.0.0-1.el8.x86_64 123/140 2020-11-20T13:55:15.050 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-modules-core-2:15.2.5-588.ga7b62340.el8 113/140 2020-11-20T13:55:15.074 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libcephfs-devel-2:15.2.5-588.ga7b62340.el8.x86_6 124/140 2020-11-20T13:55:15.203 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-2:15.2.5-588.ga7b62340.el8.x86_64 114/140 2020-11-20T13:55:15.232 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-2:15.2.5-588.ga7b62340.el8.x86_64 114/140 2020-11-20T13:55:15.387 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-rook-2:15.2.5-588.ga7b62340.el8.noarch 115/140 2020-11-20T13:55:15.401 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-rook-2:15.2.5-588.ga7b62340.el8.noarch 115/140 2020-11-20T13:55:15.453 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : elfutils-libelf-devel-0.176-5.el8.x86_64 125/140 2020-11-20T13:55:15.569 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-2:15.2.5-588.ga7b62340.el8.x86_64 116/140 2020-11-20T13:55:15.686 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libtool-2.4.6-25.el8.x86_64 117/140 2020-11-20T13:55:15.803 INFO:teuthology.orchestra.run.smithi116.stdout: Downgrading : krb5-workstation-1.17-9.el8.x86_64 126/140 2020-11-20T13:55:16.385 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : rbd-fuse-2:15.2.5-588.ga7b62340.el8.x86_64 127/140 2020-11-20T13:55:16.619 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : rbd-nbd-2:15.2.5-588.ga7b62340.el8.x86_64 128/140 2020-11-20T13:55:16.847 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-immutable-object-cache-2:15.2.5-588.ga7b623 129/140 2020-11-20T13:55:16.956 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-immutable-object-cache-2:15.2.5-588.ga7b623 129/140 2020-11-20T13:55:17.222 INFO:teuthology.orchestra.run.smithi116.stdout: Downgrading : openssl-1:1.1.1c-2.el8_1.1.x86_64 130/140 2020-11-20T13:55:17.417 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/140 2020-11-20T13:55:17.604 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : flex-2.6.1-9.el8.x86_64 132/140 2020-11-20T13:55:17.690 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 132/140 2020-11-20T13:55:17.913 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : bison-3.0.4-10.el8.x86_64 133/140 2020-11-20T13:55:17.931 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 133/140 2020-11-20T13:55:18.148 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : krb5-workstation-1.17-18.el8.x86_64 134/140 2020-11-20T13:55:18.306 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : libkadm5-1.17-18.el8.x86_64 135/140 2020-11-20T13:55:18.483 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : openssl-1:1.1.1c-15.el8.x86_64 136/140 2020-11-20T13:55:18.689 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : krb5-libs-1.17-18.el8.x86_64 137/140 2020-11-20T13:55:18.689 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 138/140 2020-11-20T13:55:18.924 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 138/140 2020-11-20T13:55:18.924 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 139/140 2020-11-20T13:55:19.175 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 139/140 2020-11-20T13:55:19.175 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : openssl-libs-1:1.1.1c-15.el8.x86_64 140/140 2020-11-20T13:55:19.825 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 117/140 2020-11-20T13:55:20.055 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: openssl-libs-1:1.1.1c-15.el8.x86_64 140/140 2020-11-20T13:55:20.056 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : krb5-libs-1.17-9.el8.x86_64 1/140 2020-11-20T13:55:20.056 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : krb5-libs-1.17-18.el8.x86_64 2/140 2020-11-20T13:55:20.056 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : krb5-workstation-1.17-9.el8.x86_64 3/140 2020-11-20T13:55:20.057 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : krb5-workstation-1.17-18.el8.x86_64 4/140 2020-11-20T13:55:20.057 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libkadm5-1.17-9.el8.x86_64 5/140 2020-11-20T13:55:20.057 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libkadm5-1.17-18.el8.x86_64 6/140 2020-11-20T13:55:20.057 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : openssl-1:1.1.1c-2.el8_1.1.x86_64 7/140 2020-11-20T13:55:20.057 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : openssl-1:1.1.1c-15.el8.x86_64 8/140 2020-11-20T13:55:20.058 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 9/140 2020-11-20T13:55:20.058 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : openssl-libs-1:1.1.1c-15.el8.x86_64 10/140 2020-11-20T13:55:20.058 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-2:15.2.5-588.ga7b62340.el8.x86_64 11/140 2020-11-20T13:55:20.058 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-base-2:15.2.5-588.ga7b62340.el8.x86_64 12/140 2020-11-20T13:55:20.058 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-common-2:15.2.5-588.ga7b62340.el8.x86_64 13/140 2020-11-20T13:55:20.059 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-fuse-2:15.2.5-588.ga7b62340.el8.x86_64 14/140 2020-11-20T13:55:20.059 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-immutable-object-cache-2:15.2.5-588.ga7b623 15/140 2020-11-20T13:55:20.059 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-mds-2:15.2.5-588.ga7b62340.el8.x86_64 16/140 2020-11-20T13:55:20.059 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-mgr-2:15.2.5-588.ga7b62340.el8.x86_64 17/140 2020-11-20T13:55:20.059 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-mon-2:15.2.5-588.ga7b62340.el8.x86_64 18/140 2020-11-20T13:55:20.060 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-osd-2:15.2.5-588.ga7b62340.el8.x86_64 19/140 2020-11-20T13:55:20.060 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-radosgw-2:15.2.5-588.ga7b62340.el8.x86_64 20/140 2020-11-20T13:55:20.060 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-selinux-2:15.2.5-588.ga7b62340.el8.x86_64 21/140 2020-11-20T13:55:20.060 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-test-2:15.2.5-588.ga7b62340.el8.x86_64 22/140 2020-11-20T13:55:20.060 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : cephadm-2:15.2.5-588.ga7b62340.el8.x86_64 23/140 2020-11-20T13:55:20.061 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libcephfs-devel-2:15.2.5-588.ga7b62340.el8.x86_6 24/140 2020-11-20T13:55:20.061 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libcephfs2-2:15.2.5-588.ga7b62340.el8.x86_64 25/140 2020-11-20T13:55:20.061 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : librados-devel-2:15.2.5-588.ga7b62340.el8.x86_64 26/140 2020-11-20T13:55:20.061 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libradosstriper1-2:15.2.5-588.ga7b62340.el8.x86_ 27/140 2020-11-20T13:55:20.062 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : librgw2-2:15.2.5-588.ga7b62340.el8.x86_64 28/140 2020-11-20T13:55:20.062 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-ceph-argparse-2:15.2.5-588.ga7b62340.el8 29/140 2020-11-20T13:55:20.062 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-ceph-common-2:15.2.5-588.ga7b62340.el8.x 30/140 2020-11-20T13:55:20.062 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-cephfs-2:15.2.5-588.ga7b62340.el8.x86_64 31/140 2020-11-20T13:55:20.062 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-rados-2:15.2.5-588.ga7b62340.el8.x86_64 32/140 2020-11-20T13:55:20.063 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-rbd-2:15.2.5-588.ga7b62340.el8.x86_64 33/140 2020-11-20T13:55:20.063 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-rgw-2:15.2.5-588.ga7b62340.el8.x86_64 34/140 2020-11-20T13:55:20.063 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : rbd-fuse-2:15.2.5-588.ga7b62340.el8.x86_64 35/140 2020-11-20T13:55:20.063 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : rbd-mirror-2:15.2.5-588.ga7b62340.el8.x86_64 36/140 2020-11-20T13:55:20.063 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : rbd-nbd-2:15.2.5-588.ga7b62340.el8.x86_64 37/140 2020-11-20T13:55:20.064 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-grafana-dashboards-2:15.2.5-588.ga7b62340.e 38/140 2020-11-20T13:55:20.064 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-mgr-cephadm-2:15.2.5-588.ga7b62340.el8.noar 39/140 2020-11-20T13:55:20.064 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-mgr-dashboard-2:15.2.5-588.ga7b62340.el8.no 40/140 2020-11-20T13:55:20.064 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-mgr-diskprediction-local-2:15.2.5-588.ga7b6 41/140 2020-11-20T13:55:20.064 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-mgr-modules-core-2:15.2.5-588.ga7b62340.el8 42/140 2020-11-20T13:55:20.065 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-mgr-rook-2:15.2.5-588.ga7b62340.el8.noarch 43/140 2020-11-20T13:55:20.065 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-prometheus-alerts-2:15.2.5-588.ga7b62340.el 44/140 2020-11-20T13:55:20.065 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : automake-1.16.1-6.el8.noarch 45/140 2020-11-20T13:55:20.065 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : bison-3.0.4-10.el8.x86_64 46/140 2020-11-20T13:55:20.065 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : flex-2.6.1-9.el8.x86_64 47/140 2020-11-20T13:55:20.066 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : jq-1.5-12.el8.x86_64 48/140 2020-11-20T13:55:20.066 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 49/140 2020-11-20T13:55:20.066 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 50/140 2020-11-20T13:55:20.066 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 51/140 2020-11-20T13:55:20.066 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : oniguruma-6.8.2-1.el8.x86_64 52/140 2020-11-20T13:55:20.067 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : openblas-0.3.3-2.el8.x86_64 53/140 2020-11-20T13:55:20.067 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : openblas-threads-0.3.3-2.el8.x86_64 54/140 2020-11-20T13:55:20.067 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 55/140 2020-11-20T13:55:20.067 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-html5lib-1:0.999999999-6.el8.noarch 56/140 2020-11-20T13:55:20.067 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 57/140 2020-11-20T13:55:20.068 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 58/140 2020-11-20T13:55:20.068 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 59/140 2020-11-20T13:55:20.068 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-lxml-4.2.3-1.el8.x86_64 60/140 2020-11-20T13:55:20.068 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 61/140 2020-11-20T13:55:20.068 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 62/140 2020-11-20T13:55:20.069 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 63/140 2020-11-20T13:55:20.069 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 64/140 2020-11-20T13:55:20.069 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 65/140 2020-11-20T13:55:20.069 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 66/140 2020-11-20T13:55:20.069 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 67/140 2020-11-20T13:55:20.070 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 68/140 2020-11-20T13:55:20.070 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-scipy-1.0.0-20.module_el8.1.0+245+c39af4 69/140 2020-11-20T13:55:20.070 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-webencodings-0.5.1-6.el8.noarch 70/140 2020-11-20T13:55:20.070 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 71/140 2020-11-20T13:55:20.070 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : socat-1.7.3.2-6.el8.x86_64 72/140 2020-11-20T13:55:20.071 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : elfutils-libelf-devel-0.176-5.el8.x86_64 73/140 2020-11-20T13:55:20.071 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 74/140 2020-11-20T13:55:20.071 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : fuse-common-3.2.1-12.el8.x86_64 75/140 2020-11-20T13:55:20.071 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 76/140 2020-11-20T13:55:20.071 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : krb5-devel-1.17-9.el8.x86_64 77/140 2020-11-20T13:55:20.071 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 78/140 2020-11-20T13:55:20.072 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libcom_err-devel-1.44.6-3.el8.x86_64 79/140 2020-11-20T13:55:20.072 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libgfortran-8.3.1-4.5.el8.x86_64 80/140 2020-11-20T13:55:20.072 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libquadmath-8.3.1-4.5.el8.x86_64 81/140 2020-11-20T13:55:20.072 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : librabbitmq-0.9.0-1.el8.x86_64 82/140 2020-11-20T13:55:20.072 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libselinux-devel-2.9-2.1.el8.x86_64 83/140 2020-11-20T13:55:20.073 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libsepol-devel-2.9-1.el8.x86_64 84/140 2020-11-20T13:55:20.073 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libuuid-devel-2.32.1-17.el8.x86_64 85/140 2020-11-20T13:55:20.073 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 86/140 2020-11-20T13:55:20.073 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libxslt-1.1.32-3.el8.x86_64 87/140 2020-11-20T13:55:20.073 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : openssl-devel-1:1.1.1c-2.el8_1.1.x86_64 88/140 2020-11-20T13:55:20.074 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : pcre2-devel-10.32-1.el8.x86_64 89/140 2020-11-20T13:55:20.074 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : pcre2-utf16-10.32-1.el8.x86_64 90/140 2020-11-20T13:55:20.074 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : pcre2-utf32-10.32-1.el8.x86_64 91/140 2020-11-20T13:55:20.074 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-chardet-3.0.4-7.el8.noarch 92/140 2020-11-20T13:55:20.074 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 93/140 2020-11-20T13:55:20.075 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 94/140 2020-11-20T13:55:20.075 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-pysocks-1.6.8-3.el8.noarch 95/140 2020-11-20T13:55:20.075 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-requests-2.20.0-2.1.el8_1.noarch 96/140 2020-11-20T13:55:20.075 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 97/140 2020-11-20T13:55:20.075 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-urllib3-1.24.2-2.el8.noarch 98/140 2020-11-20T13:55:20.076 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : xfsprogs-devel-5.0.0-1.el8.x86_64 99/140 2020-11-20T13:55:20.076 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : zlib-devel-1.2.11-10.el8.x86_64 100/140 2020-11-20T13:55:20.076 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : fmt-6.2.1-1.el8.x86_64 101/140 2020-11-20T13:55:20.076 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : gperftools-libs-2.7-6.el8.x86_64 102/140 2020-11-20T13:55:20.076 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : leveldb-1.22-1.el8.x86_64 103/140 2020-11-20T13:55:20.077 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 104/140 2020-11-20T13:55:20.077 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 105/140 2020-11-20T13:55:20.077 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 106/140 2020-11-20T13:55:20.077 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 107/140 2020-11-20T13:55:20.077 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 108/140 2020-11-20T13:55:20.078 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 109/140 2020-11-20T13:55:20.078 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 110/140 2020-11-20T13:55:20.078 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-cheroot-8.2.1-1.el8.noarch 111/140 2020-11-20T13:55:20.078 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 112/140 2020-11-20T13:55:20.078 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-cssselect-0.9.2-10.el8.noarch 113/140 2020-11-20T13:55:20.078 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-execnet-1.7.1-1.el8.noarch 114/140 2020-11-20T13:55:20.079 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 115/140 2020-11-20T13:55:20.079 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 116/140 2020-11-20T13:55:20.079 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 117/140 2020-11-20T13:55:20.079 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-kubernetes-11.0.0-2.el8.noarch 118/140 2020-11-20T13:55:20.079 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 119/140 2020-11-20T13:55:20.080 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 120/140 2020-11-20T13:55:20.080 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 121/140 2020-11-20T13:55:20.080 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-portend-2.6-1.el8.noarch 122/140 2020-11-20T13:55:20.080 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-remoto-1.1.4-4.el8.noarch 123/140 2020-11-20T13:55:20.080 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 124/140 2020-11-20T13:55:20.081 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 125/140 2020-11-20T13:55:20.081 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-rsa-3.4.2-15.el8.noarch 126/140 2020-11-20T13:55:20.081 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 127/140 2020-11-20T13:55:20.081 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 128/140 2020-11-20T13:55:20.081 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 129/140 2020-11-20T13:55:20.082 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 130/140 2020-11-20T13:55:20.082 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 131/140 2020-11-20T13:55:20.082 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 132/140 2020-11-20T13:55:20.082 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 133/140 2020-11-20T13:55:20.082 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 134/140 2020-11-20T13:55:20.083 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 135/140 2020-11-20T13:55:20.083 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : xmlstarlet-1.6.1-11.el8.x86_64 136/140 2020-11-20T13:55:20.083 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : librados2-2:15.2.5-588.ga7b62340.el8.x86_64 137/140 2020-11-20T13:55:20.083 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 138/140 2020-11-20T13:55:20.083 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : librbd1-2:15.2.5-588.ga7b62340.el8.x86_64 139/140 2020-11-20T13:55:21.036 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-test-2:15.2.5-588.ga7b62340.el8.x86_64 118/140 2020-11-20T13:55:21.219 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-radosgw-2:15.2.5-588.ga7b62340.el8.x86_64 119/140 2020-11-20T13:55:21.590 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-radosgw-2:15.2.5-588.ga7b62340.el8.x86_64 119/140 2020-11-20T13:55:21.744 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-mirror-2:15.2.5-588.ga7b62340.el8.x86_64 120/140 2020-11-20T13:55:21.776 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 140/140 2020-11-20T13:55:21.777 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T13:55:21.777 INFO:teuthology.orchestra.run.smithi116.stdout:Upgraded: 2020-11-20T13:55:21.778 INFO:teuthology.orchestra.run.smithi116.stdout: librados2-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:21.778 INFO:teuthology.orchestra.run.smithi116.stdout: librbd1-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:21.778 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T13:55:21.779 INFO:teuthology.orchestra.run.smithi116.stdout:Downgraded: 2020-11-20T13:55:21.779 INFO:teuthology.orchestra.run.smithi116.stdout: krb5-libs-1.17-9.el8.x86_64 krb5-workstation-1.17-9.el8.x86_64 2020-11-20T13:55:21.780 INFO:teuthology.orchestra.run.smithi116.stdout: libkadm5-1.17-9.el8.x86_64 openssl-1:1.1.1c-2.el8_1.1.x86_64 2020-11-20T13:55:21.781 INFO:teuthology.orchestra.run.smithi116.stdout: openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 2020-11-20T13:55:21.781 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T13:55:21.781 INFO:teuthology.orchestra.run.smithi116.stdout:Installed: 2020-11-20T13:55:21.782 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:21.782 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-base-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:21.782 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-fuse-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:21.782 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-immutable-object-cache-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:21.782 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:21.783 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-radosgw-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:21.783 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-test-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:21.783 INFO:teuthology.orchestra.run.smithi116.stdout: cephadm-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:21.783 INFO:teuthology.orchestra.run.smithi116.stdout: libcephfs-devel-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:21.783 INFO:teuthology.orchestra.run.smithi116.stdout: libcephfs2-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:21.784 INFO:teuthology.orchestra.run.smithi116.stdout: librados-devel-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:21.784 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cephfs-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:21.784 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rados-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:21.784 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rbd-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:21.784 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rgw-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:21.785 INFO:teuthology.orchestra.run.smithi116.stdout: rbd-fuse-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:21.785 INFO:teuthology.orchestra.run.smithi116.stdout: rbd-mirror-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:21.785 INFO:teuthology.orchestra.run.smithi116.stdout: rbd-nbd-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:21.786 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-cephadm-2:15.2.5-588.ga7b62340.el8.noarch 2020-11-20T13:55:21.786 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-dashboard-2:15.2.5-588.ga7b62340.el8.noarch 2020-11-20T13:55:21.786 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-diskprediction-local-2:15.2.5-588.ga7b62340.el8.noarch 2020-11-20T13:55:21.786 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-rook-2:15.2.5-588.ga7b62340.el8.noarch 2020-11-20T13:55:21.786 INFO:teuthology.orchestra.run.smithi116.stdout: bison-3.0.4-10.el8.x86_64 2020-11-20T13:55:21.787 INFO:teuthology.orchestra.run.smithi116.stdout: flex-2.6.1-9.el8.x86_64 2020-11-20T13:55:21.787 INFO:teuthology.orchestra.run.smithi116.stdout: libtool-2.4.6-25.el8.x86_64 2020-11-20T13:55:21.787 INFO:teuthology.orchestra.run.smithi116.stdout: elfutils-libelf-devel-0.176-5.el8.x86_64 2020-11-20T13:55:21.788 INFO:teuthology.orchestra.run.smithi116.stdout: libaio-devel-0.3.112-1.el8.x86_64 2020-11-20T13:55:21.788 INFO:teuthology.orchestra.run.smithi116.stdout: libuuid-devel-2.32.1-17.el8.x86_64 2020-11-20T13:55:21.788 INFO:teuthology.orchestra.run.smithi116.stdout: openssl-devel-1:1.1.1c-2.el8_1.1.x86_64 2020-11-20T13:55:21.788 INFO:teuthology.orchestra.run.smithi116.stdout: xfsprogs-devel-5.0.0-1.el8.x86_64 2020-11-20T13:55:21.788 INFO:teuthology.orchestra.run.smithi116.stdout: python3-html5lib-1:0.999999999-6.el8.noarch 2020-11-20T13:55:21.789 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cssselect-0.9.2-10.el8.noarch 2020-11-20T13:55:21.789 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-common-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:21.789 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mds-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:21.789 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mon-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:21.789 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-osd-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:21.790 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-selinux-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:21.790 INFO:teuthology.orchestra.run.smithi116.stdout: libradosstriper1-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:21.790 INFO:teuthology.orchestra.run.smithi116.stdout: librgw2-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:21.790 INFO:teuthology.orchestra.run.smithi116.stdout: python3-ceph-argparse-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:21.791 INFO:teuthology.orchestra.run.smithi116.stdout: python3-ceph-common-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:21.791 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-grafana-dashboards-2:15.2.5-588.ga7b62340.el8.noarch 2020-11-20T13:55:21.791 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-modules-core-2:15.2.5-588.ga7b62340.el8.noarch 2020-11-20T13:55:21.791 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-prometheus-alerts-2:15.2.5-588.ga7b62340.el8.noarch 2020-11-20T13:55:21.791 INFO:teuthology.orchestra.run.smithi116.stdout: automake-1.16.1-6.el8.noarch 2020-11-20T13:55:21.792 INFO:teuthology.orchestra.run.smithi116.stdout: jq-1.5-12.el8.x86_64 2020-11-20T13:55:21.792 INFO:teuthology.orchestra.run.smithi116.stdout: librdkafka-0.11.4-1.el8.x86_64 2020-11-20T13:55:21.792 INFO:teuthology.orchestra.run.smithi116.stdout: lttng-ust-2.8.1-11.el8.x86_64 2020-11-20T13:55:21.792 INFO:teuthology.orchestra.run.smithi116.stdout: oniguruma-6.8.2-1.el8.x86_64 2020-11-20T13:55:21.792 INFO:teuthology.orchestra.run.smithi116.stdout: openblas-0.3.3-2.el8.x86_64 2020-11-20T13:55:21.792 INFO:teuthology.orchestra.run.smithi116.stdout: openblas-threads-0.3.3-2.el8.x86_64 2020-11-20T13:55:21.793 INFO:teuthology.orchestra.run.smithi116.stdout: python3-babel-2.5.1-5.el8.noarch 2020-11-20T13:55:21.793 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2020-11-20T13:55:21.793 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jsonpatch-1.21-2.el8.noarch 2020-11-20T13:55:21.793 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jsonpointer-1.10-11.el8.noarch 2020-11-20T13:55:21.794 INFO:teuthology.orchestra.run.smithi116.stdout: python3-lxml-4.2.3-1.el8.x86_64 2020-11-20T13:55:21.794 INFO:teuthology.orchestra.run.smithi116.stdout: python3-mako-1.0.6-13.el8.noarch 2020-11-20T13:55:21.794 INFO:teuthology.orchestra.run.smithi116.stdout: python3-markupsafe-0.23-19.el8.x86_64 2020-11-20T13:55:21.794 INFO:teuthology.orchestra.run.smithi116.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2020-11-20T13:55:21.794 INFO:teuthology.orchestra.run.smithi116.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2020-11-20T13:55:21.795 INFO:teuthology.orchestra.run.smithi116.stdout: python3-prettytable-0.7.2-14.el8.noarch 2020-11-20T13:55:21.795 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2020-11-20T13:55:21.795 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2020-11-20T13:55:21.795 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pytz-2017.2-9.el8.noarch 2020-11-20T13:55:21.795 INFO:teuthology.orchestra.run.smithi116.stdout: python3-scipy-1.0.0-20.module_el8.1.0+245+c39af44f.x86_64 2020-11-20T13:55:21.795 INFO:teuthology.orchestra.run.smithi116.stdout: python3-webencodings-0.5.1-6.el8.noarch 2020-11-20T13:55:21.796 INFO:teuthology.orchestra.run.smithi116.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2020-11-20T13:55:21.796 INFO:teuthology.orchestra.run.smithi116.stdout: socat-1.7.3.2-6.el8.x86_64 2020-11-20T13:55:21.796 INFO:teuthology.orchestra.run.smithi116.stdout: fuse-2.9.7-12.el8.x86_64 2020-11-20T13:55:21.796 INFO:teuthology.orchestra.run.smithi116.stdout: fuse-common-3.2.1-12.el8.x86_64 2020-11-20T13:55:21.797 INFO:teuthology.orchestra.run.smithi116.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2020-11-20T13:55:21.797 INFO:teuthology.orchestra.run.smithi116.stdout: krb5-devel-1.17-9.el8.x86_64 2020-11-20T13:55:21.798 INFO:teuthology.orchestra.run.smithi116.stdout: libcom_err-devel-1.44.6-3.el8.x86_64 2020-11-20T13:55:21.798 INFO:teuthology.orchestra.run.smithi116.stdout: libgfortran-8.3.1-4.5.el8.x86_64 2020-11-20T13:55:21.799 INFO:teuthology.orchestra.run.smithi116.stdout: libquadmath-8.3.1-4.5.el8.x86_64 2020-11-20T13:55:21.799 INFO:teuthology.orchestra.run.smithi116.stdout: librabbitmq-0.9.0-1.el8.x86_64 2020-11-20T13:55:21.799 INFO:teuthology.orchestra.run.smithi116.stdout: libselinux-devel-2.9-2.1.el8.x86_64 2020-11-20T13:55:21.799 INFO:teuthology.orchestra.run.smithi116.stdout: libsepol-devel-2.9-1.el8.x86_64 2020-11-20T13:55:21.800 INFO:teuthology.orchestra.run.smithi116.stdout: libverto-devel-0.3.0-5.el8.x86_64 2020-11-20T13:55:21.800 INFO:teuthology.orchestra.run.smithi116.stdout: libxslt-1.1.32-3.el8.x86_64 2020-11-20T13:55:21.800 INFO:teuthology.orchestra.run.smithi116.stdout: pcre2-devel-10.32-1.el8.x86_64 2020-11-20T13:55:21.800 INFO:teuthology.orchestra.run.smithi116.stdout: pcre2-utf16-10.32-1.el8.x86_64 2020-11-20T13:55:21.801 INFO:teuthology.orchestra.run.smithi116.stdout: pcre2-utf32-10.32-1.el8.x86_64 2020-11-20T13:55:21.801 INFO:teuthology.orchestra.run.smithi116.stdout: python3-chardet-3.0.4-7.el8.noarch 2020-11-20T13:55:21.801 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jwt-1.6.1-2.el8.noarch 2020-11-20T13:55:21.801 INFO:teuthology.orchestra.run.smithi116.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2020-11-20T13:55:21.801 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pysocks-1.6.8-3.el8.noarch 2020-11-20T13:55:21.802 INFO:teuthology.orchestra.run.smithi116.stdout: python3-requests-2.20.0-2.1.el8_1.noarch 2020-11-20T13:55:21.802 INFO:teuthology.orchestra.run.smithi116.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2020-11-20T13:55:21.802 INFO:teuthology.orchestra.run.smithi116.stdout: python3-urllib3-1.24.2-2.el8.noarch 2020-11-20T13:55:21.802 INFO:teuthology.orchestra.run.smithi116.stdout: zlib-devel-1.2.11-10.el8.x86_64 2020-11-20T13:55:21.802 INFO:teuthology.orchestra.run.smithi116.stdout: fmt-6.2.1-1.el8.x86_64 2020-11-20T13:55:21.803 INFO:teuthology.orchestra.run.smithi116.stdout: gperftools-libs-2.7-6.el8.x86_64 2020-11-20T13:55:21.803 INFO:teuthology.orchestra.run.smithi116.stdout: leveldb-1.22-1.el8.x86_64 2020-11-20T13:55:21.803 INFO:teuthology.orchestra.run.smithi116.stdout: liboath-2.6.2-3.el8.x86_64 2020-11-20T13:55:21.804 INFO:teuthology.orchestra.run.smithi116.stdout: libunwind-1.3.1-3.el8.x86_64 2020-11-20T13:55:21.804 INFO:teuthology.orchestra.run.smithi116.stdout: python3-apipkg-1.5-6.el8.noarch 2020-11-20T13:55:21.804 INFO:teuthology.orchestra.run.smithi116.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2020-11-20T13:55:21.804 INFO:teuthology.orchestra.run.smithi116.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2020-11-20T13:55:21.805 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cachetools-3.1.1-4.el8.noarch 2020-11-20T13:55:21.805 INFO:teuthology.orchestra.run.smithi116.stdout: python3-certifi-2018.10.15-7.el8.noarch 2020-11-20T13:55:21.805 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cheroot-8.2.1-1.el8.noarch 2020-11-20T13:55:21.805 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2020-11-20T13:55:21.805 INFO:teuthology.orchestra.run.smithi116.stdout: python3-execnet-1.7.1-1.el8.noarch 2020-11-20T13:55:21.806 INFO:teuthology.orchestra.run.smithi116.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2020-11-20T13:55:21.806 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco-6.2-6.el8.noarch 2020-11-20T13:55:21.806 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2020-11-20T13:55:21.807 INFO:teuthology.orchestra.run.smithi116.stdout: python3-kubernetes-11.0.0-2.el8.noarch 2020-11-20T13:55:21.807 INFO:teuthology.orchestra.run.smithi116.stdout: python3-logutils-0.3.5-11.el8.noarch 2020-11-20T13:55:21.807 INFO:teuthology.orchestra.run.smithi116.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2020-11-20T13:55:21.807 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pecan-1.3.2-9.el8.noarch 2020-11-20T13:55:21.808 INFO:teuthology.orchestra.run.smithi116.stdout: python3-portend-2.6-1.el8.noarch 2020-11-20T13:55:21.808 INFO:teuthology.orchestra.run.smithi116.stdout: python3-remoto-1.1.4-4.el8.noarch 2020-11-20T13:55:21.808 INFO:teuthology.orchestra.run.smithi116.stdout: python3-repoze-lru-0.7-6.el8.noarch 2020-11-20T13:55:21.808 INFO:teuthology.orchestra.run.smithi116.stdout: python3-routes-2.4.1-12.el8.noarch 2020-11-20T13:55:21.809 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rsa-3.4.2-15.el8.noarch 2020-11-20T13:55:21.809 INFO:teuthology.orchestra.run.smithi116.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2020-11-20T13:55:21.809 INFO:teuthology.orchestra.run.smithi116.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2020-11-20T13:55:21.809 INFO:teuthology.orchestra.run.smithi116.stdout: python3-tempora-1.14.1-5.el8.noarch 2020-11-20T13:55:21.809 INFO:teuthology.orchestra.run.smithi116.stdout: python3-trustme-0.6.0-4.el8.noarch 2020-11-20T13:55:21.810 INFO:teuthology.orchestra.run.smithi116.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2020-11-20T13:55:21.810 INFO:teuthology.orchestra.run.smithi116.stdout: python3-webob-1.8.5-1.el8.1.noarch 2020-11-20T13:55:21.810 INFO:teuthology.orchestra.run.smithi116.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2020-11-20T13:55:21.810 INFO:teuthology.orchestra.run.smithi116.stdout: python3-webtest-2.0.33-1.el8.noarch 2020-11-20T13:55:21.810 INFO:teuthology.orchestra.run.smithi116.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2020-11-20T13:55:21.811 INFO:teuthology.orchestra.run.smithi116.stdout: xmlstarlet-1.6.1-11.el8.x86_64 2020-11-20T13:55:21.811 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T13:55:21.812 INFO:teuthology.orchestra.run.smithi116.stdout:Complete! 2020-11-20T13:55:21.817 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: rbd-mirror-2:15.2.5-588.ga7b62340.el8.x86_64 120/140 2020-11-20T13:55:21.931 DEBUG:teuthology.parallel:result is None 2020-11-20T13:55:22.880 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-fuse-2:15.2.5-588.ga7b62340.el8.x86_64 121/140 2020-11-20T13:55:23.122 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : openssl-devel-1:1.1.1c-2.el8_1.1.x86_64 122/140 2020-11-20T13:55:23.313 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : xfsprogs-devel-5.0.0-1.el8.x86_64 123/140 2020-11-20T13:55:23.522 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephfs-devel-2:15.2.5-588.ga7b62340.el8.x86_6 124/140 2020-11-20T13:55:23.882 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : elfutils-libelf-devel-0.176-5.el8.x86_64 125/140 2020-11-20T13:55:24.343 INFO:teuthology.orchestra.run.smithi040.stdout: Downgrading : krb5-workstation-1.17-9.el8.x86_64 126/140 2020-11-20T13:55:24.533 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-fuse-2:15.2.5-588.ga7b62340.el8.x86_64 127/140 2020-11-20T13:55:24.706 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-nbd-2:15.2.5-588.ga7b62340.el8.x86_64 128/140 2020-11-20T13:55:24.902 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-immutable-object-cache-2:15.2.5-588.ga7b623 129/140 2020-11-20T13:55:25.014 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-immutable-object-cache-2:15.2.5-588.ga7b623 129/140 2020-11-20T13:55:25.228 INFO:teuthology.orchestra.run.smithi040.stdout: Downgrading : openssl-1:1.1.1c-2.el8_1.1.x86_64 130/140 2020-11-20T13:55:25.481 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/140 2020-11-20T13:55:25.627 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flex-2.6.1-9.el8.x86_64 132/140 2020-11-20T13:55:25.728 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 132/140 2020-11-20T13:55:25.877 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : bison-3.0.4-10.el8.x86_64 133/140 2020-11-20T13:55:25.897 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 133/140 2020-11-20T13:55:26.095 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : krb5-workstation-1.17-18.el8.x86_64 134/140 2020-11-20T13:55:26.206 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : libkadm5-1.17-18.el8.x86_64 135/140 2020-11-20T13:55:26.372 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : openssl-1:1.1.1c-15.el8.x86_64 136/140 2020-11-20T13:55:26.528 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : krb5-libs-1.17-18.el8.x86_64 137/140 2020-11-20T13:55:26.528 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 138/140 2020-11-20T13:55:26.772 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 138/140 2020-11-20T13:55:26.772 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 139/140 2020-11-20T13:55:27.002 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 139/140 2020-11-20T13:55:27.003 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : openssl-libs-1:1.1.1c-15.el8.x86_64 140/140 2020-11-20T13:55:27.919 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: openssl-libs-1:1.1.1c-15.el8.x86_64 140/140 2020-11-20T13:55:27.920 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : krb5-libs-1.17-9.el8.x86_64 1/140 2020-11-20T13:55:27.920 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : krb5-libs-1.17-18.el8.x86_64 2/140 2020-11-20T13:55:27.920 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : krb5-workstation-1.17-9.el8.x86_64 3/140 2020-11-20T13:55:27.920 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : krb5-workstation-1.17-18.el8.x86_64 4/140 2020-11-20T13:55:27.920 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libkadm5-1.17-9.el8.x86_64 5/140 2020-11-20T13:55:27.921 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libkadm5-1.17-18.el8.x86_64 6/140 2020-11-20T13:55:27.921 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openssl-1:1.1.1c-2.el8_1.1.x86_64 7/140 2020-11-20T13:55:27.921 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openssl-1:1.1.1c-15.el8.x86_64 8/140 2020-11-20T13:55:27.921 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 9/140 2020-11-20T13:55:27.922 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openssl-libs-1:1.1.1c-15.el8.x86_64 10/140 2020-11-20T13:55:27.922 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-2:15.2.5-588.ga7b62340.el8.x86_64 11/140 2020-11-20T13:55:27.922 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-base-2:15.2.5-588.ga7b62340.el8.x86_64 12/140 2020-11-20T13:55:27.922 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-common-2:15.2.5-588.ga7b62340.el8.x86_64 13/140 2020-11-20T13:55:27.922 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-fuse-2:15.2.5-588.ga7b62340.el8.x86_64 14/140 2020-11-20T13:55:27.923 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-immutable-object-cache-2:15.2.5-588.ga7b623 15/140 2020-11-20T13:55:27.923 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mds-2:15.2.5-588.ga7b62340.el8.x86_64 16/140 2020-11-20T13:55:27.923 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-2:15.2.5-588.ga7b62340.el8.x86_64 17/140 2020-11-20T13:55:27.923 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mon-2:15.2.5-588.ga7b62340.el8.x86_64 18/140 2020-11-20T13:55:27.923 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-osd-2:15.2.5-588.ga7b62340.el8.x86_64 19/140 2020-11-20T13:55:27.924 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-radosgw-2:15.2.5-588.ga7b62340.el8.x86_64 20/140 2020-11-20T13:55:27.924 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-selinux-2:15.2.5-588.ga7b62340.el8.x86_64 21/140 2020-11-20T13:55:27.924 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-test-2:15.2.5-588.ga7b62340.el8.x86_64 22/140 2020-11-20T13:55:27.924 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : cephadm-2:15.2.5-588.ga7b62340.el8.x86_64 23/140 2020-11-20T13:55:27.924 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephfs-devel-2:15.2.5-588.ga7b62340.el8.x86_6 24/140 2020-11-20T13:55:27.925 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephfs2-2:15.2.5-588.ga7b62340.el8.x86_64 25/140 2020-11-20T13:55:27.925 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados-devel-2:15.2.5-588.ga7b62340.el8.x86_64 26/140 2020-11-20T13:55:27.925 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libradosstriper1-2:15.2.5-588.ga7b62340.el8.x86_ 27/140 2020-11-20T13:55:27.925 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librgw2-2:15.2.5-588.ga7b62340.el8.x86_64 28/140 2020-11-20T13:55:27.926 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-ceph-argparse-2:15.2.5-588.ga7b62340.el8 29/140 2020-11-20T13:55:27.926 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-ceph-common-2:15.2.5-588.ga7b62340.el8.x 30/140 2020-11-20T13:55:27.926 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cephfs-2:15.2.5-588.ga7b62340.el8.x86_64 31/140 2020-11-20T13:55:27.926 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rados-2:15.2.5-588.ga7b62340.el8.x86_64 32/140 2020-11-20T13:55:27.926 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rbd-2:15.2.5-588.ga7b62340.el8.x86_64 33/140 2020-11-20T13:55:27.927 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rgw-2:15.2.5-588.ga7b62340.el8.x86_64 34/140 2020-11-20T13:55:27.927 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-fuse-2:15.2.5-588.ga7b62340.el8.x86_64 35/140 2020-11-20T13:55:27.927 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-mirror-2:15.2.5-588.ga7b62340.el8.x86_64 36/140 2020-11-20T13:55:27.927 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-nbd-2:15.2.5-588.ga7b62340.el8.x86_64 37/140 2020-11-20T13:55:27.927 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-grafana-dashboards-2:15.2.5-588.ga7b62340.e 38/140 2020-11-20T13:55:27.928 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-cephadm-2:15.2.5-588.ga7b62340.el8.noar 39/140 2020-11-20T13:55:27.928 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-dashboard-2:15.2.5-588.ga7b62340.el8.no 40/140 2020-11-20T13:55:27.928 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-diskprediction-local-2:15.2.5-588.ga7b6 41/140 2020-11-20T13:55:27.928 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-modules-core-2:15.2.5-588.ga7b62340.el8 42/140 2020-11-20T13:55:27.928 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-rook-2:15.2.5-588.ga7b62340.el8.noarch 43/140 2020-11-20T13:55:27.929 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-prometheus-alerts-2:15.2.5-588.ga7b62340.el 44/140 2020-11-20T13:55:27.929 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : automake-1.16.1-6.el8.noarch 45/140 2020-11-20T13:55:27.929 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : bison-3.0.4-10.el8.x86_64 46/140 2020-11-20T13:55:27.929 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flex-2.6.1-9.el8.x86_64 47/140 2020-11-20T13:55:27.930 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : jq-1.5-12.el8.x86_64 48/140 2020-11-20T13:55:27.930 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 49/140 2020-11-20T13:55:27.930 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 50/140 2020-11-20T13:55:27.930 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 51/140 2020-11-20T13:55:27.930 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : oniguruma-6.8.2-1.el8.x86_64 52/140 2020-11-20T13:55:27.931 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openblas-0.3.3-2.el8.x86_64 53/140 2020-11-20T13:55:27.931 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openblas-threads-0.3.3-2.el8.x86_64 54/140 2020-11-20T13:55:27.931 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 55/140 2020-11-20T13:55:27.931 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-html5lib-1:0.999999999-6.el8.noarch 56/140 2020-11-20T13:55:27.931 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 57/140 2020-11-20T13:55:27.932 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 58/140 2020-11-20T13:55:27.932 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 59/140 2020-11-20T13:55:27.932 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-lxml-4.2.3-1.el8.x86_64 60/140 2020-11-20T13:55:27.932 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 61/140 2020-11-20T13:55:27.932 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 62/140 2020-11-20T13:55:27.933 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 63/140 2020-11-20T13:55:27.933 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 64/140 2020-11-20T13:55:27.933 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 65/140 2020-11-20T13:55:27.933 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 66/140 2020-11-20T13:55:27.933 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 67/140 2020-11-20T13:55:27.934 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 68/140 2020-11-20T13:55:27.934 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-scipy-1.0.0-20.module_el8.1.0+245+c39af4 69/140 2020-11-20T13:55:27.934 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-webencodings-0.5.1-6.el8.noarch 70/140 2020-11-20T13:55:27.934 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 71/140 2020-11-20T13:55:27.934 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : socat-1.7.3.2-6.el8.x86_64 72/140 2020-11-20T13:55:27.935 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : elfutils-libelf-devel-0.176-5.el8.x86_64 73/140 2020-11-20T13:55:27.935 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 74/140 2020-11-20T13:55:27.935 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : fuse-common-3.2.1-12.el8.x86_64 75/140 2020-11-20T13:55:27.935 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 76/140 2020-11-20T13:55:27.935 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : krb5-devel-1.17-9.el8.x86_64 77/140 2020-11-20T13:55:27.936 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 78/140 2020-11-20T13:55:27.936 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcom_err-devel-1.44.6-3.el8.x86_64 79/140 2020-11-20T13:55:27.936 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libgfortran-8.3.1-4.5.el8.x86_64 80/140 2020-11-20T13:55:27.936 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libquadmath-8.3.1-4.5.el8.x86_64 81/140 2020-11-20T13:55:27.937 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librabbitmq-0.9.0-1.el8.x86_64 82/140 2020-11-20T13:55:27.937 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libselinux-devel-2.9-2.1.el8.x86_64 83/140 2020-11-20T13:55:27.937 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libsepol-devel-2.9-1.el8.x86_64 84/140 2020-11-20T13:55:27.937 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libuuid-devel-2.32.1-17.el8.x86_64 85/140 2020-11-20T13:55:27.937 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 86/140 2020-11-20T13:55:27.938 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libxslt-1.1.32-3.el8.x86_64 87/140 2020-11-20T13:55:27.938 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openssl-devel-1:1.1.1c-2.el8_1.1.x86_64 88/140 2020-11-20T13:55:27.938 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : pcre2-devel-10.32-1.el8.x86_64 89/140 2020-11-20T13:55:27.938 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : pcre2-utf16-10.32-1.el8.x86_64 90/140 2020-11-20T13:55:27.939 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : pcre2-utf32-10.32-1.el8.x86_64 91/140 2020-11-20T13:55:27.939 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-chardet-3.0.4-7.el8.noarch 92/140 2020-11-20T13:55:27.939 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 93/140 2020-11-20T13:55:27.939 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 94/140 2020-11-20T13:55:27.939 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pysocks-1.6.8-3.el8.noarch 95/140 2020-11-20T13:55:27.939 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-requests-2.20.0-2.1.el8_1.noarch 96/140 2020-11-20T13:55:27.940 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 97/140 2020-11-20T13:55:27.940 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-urllib3-1.24.2-2.el8.noarch 98/140 2020-11-20T13:55:27.940 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : xfsprogs-devel-5.0.0-1.el8.x86_64 99/140 2020-11-20T13:55:27.940 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : zlib-devel-1.2.11-10.el8.x86_64 100/140 2020-11-20T13:55:27.940 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : fmt-6.2.1-1.el8.x86_64 101/140 2020-11-20T13:55:27.941 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : gperftools-libs-2.7-6.el8.x86_64 102/140 2020-11-20T13:55:27.941 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : leveldb-1.22-1.el8.x86_64 103/140 2020-11-20T13:55:27.941 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 104/140 2020-11-20T13:55:27.941 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 105/140 2020-11-20T13:55:27.942 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 106/140 2020-11-20T13:55:27.942 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 107/140 2020-11-20T13:55:27.942 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 108/140 2020-11-20T13:55:27.942 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 109/140 2020-11-20T13:55:27.942 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 110/140 2020-11-20T13:55:27.943 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cheroot-8.2.1-1.el8.noarch 111/140 2020-11-20T13:55:27.943 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 112/140 2020-11-20T13:55:27.943 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cssselect-0.9.2-10.el8.noarch 113/140 2020-11-20T13:55:27.943 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-execnet-1.7.1-1.el8.noarch 114/140 2020-11-20T13:55:27.943 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 115/140 2020-11-20T13:55:27.944 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 116/140 2020-11-20T13:55:27.944 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 117/140 2020-11-20T13:55:27.944 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-kubernetes-11.0.0-2.el8.noarch 118/140 2020-11-20T13:55:27.944 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 119/140 2020-11-20T13:55:27.944 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 120/140 2020-11-20T13:55:27.945 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 121/140 2020-11-20T13:55:27.945 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-portend-2.6-1.el8.noarch 122/140 2020-11-20T13:55:27.945 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-remoto-1.1.4-4.el8.noarch 123/140 2020-11-20T13:55:27.945 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 124/140 2020-11-20T13:55:27.945 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 125/140 2020-11-20T13:55:27.946 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rsa-3.4.2-15.el8.noarch 126/140 2020-11-20T13:55:27.946 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 127/140 2020-11-20T13:55:27.946 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 128/140 2020-11-20T13:55:27.946 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 129/140 2020-11-20T13:55:27.946 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 130/140 2020-11-20T13:55:27.947 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 131/140 2020-11-20T13:55:27.947 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 132/140 2020-11-20T13:55:27.947 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 133/140 2020-11-20T13:55:27.947 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 134/140 2020-11-20T13:55:27.947 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 135/140 2020-11-20T13:55:27.948 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : xmlstarlet-1.6.1-11.el8.x86_64 136/140 2020-11-20T13:55:27.948 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados2-2:15.2.5-588.ga7b62340.el8.x86_64 137/140 2020-11-20T13:55:27.948 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 138/140 2020-11-20T13:55:27.948 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librbd1-2:15.2.5-588.ga7b62340.el8.x86_64 139/140 2020-11-20T13:55:29.669 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 140/140 2020-11-20T13:55:29.669 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:55:29.669 INFO:teuthology.orchestra.run.smithi040.stdout:Upgraded: 2020-11-20T13:55:29.669 INFO:teuthology.orchestra.run.smithi040.stdout: librados2-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:29.670 INFO:teuthology.orchestra.run.smithi040.stdout: librbd1-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:29.670 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:55:29.670 INFO:teuthology.orchestra.run.smithi040.stdout:Downgraded: 2020-11-20T13:55:29.670 INFO:teuthology.orchestra.run.smithi040.stdout: krb5-libs-1.17-9.el8.x86_64 krb5-workstation-1.17-9.el8.x86_64 2020-11-20T13:55:29.670 INFO:teuthology.orchestra.run.smithi040.stdout: libkadm5-1.17-9.el8.x86_64 openssl-1:1.1.1c-2.el8_1.1.x86_64 2020-11-20T13:55:29.671 INFO:teuthology.orchestra.run.smithi040.stdout: openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 2020-11-20T13:55:29.671 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:55:29.671 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2020-11-20T13:55:29.671 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:29.672 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-base-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:29.672 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-fuse-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:29.672 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-immutable-object-cache-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:29.672 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:29.673 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-radosgw-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:29.673 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-test-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:29.673 INFO:teuthology.orchestra.run.smithi040.stdout: cephadm-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:29.673 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs-devel-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:29.674 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs2-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:29.674 INFO:teuthology.orchestra.run.smithi040.stdout: librados-devel-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:29.674 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cephfs-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:29.674 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rados-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:29.675 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rbd-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:29.675 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rgw-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:29.675 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-fuse-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:29.675 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-mirror-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:29.676 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-nbd-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:29.676 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-cephadm-2:15.2.5-588.ga7b62340.el8.noarch 2020-11-20T13:55:29.676 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-dashboard-2:15.2.5-588.ga7b62340.el8.noarch 2020-11-20T13:55:29.676 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-diskprediction-local-2:15.2.5-588.ga7b62340.el8.noarch 2020-11-20T13:55:29.677 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-rook-2:15.2.5-588.ga7b62340.el8.noarch 2020-11-20T13:55:29.677 INFO:teuthology.orchestra.run.smithi040.stdout: bison-3.0.4-10.el8.x86_64 2020-11-20T13:55:29.677 INFO:teuthology.orchestra.run.smithi040.stdout: flex-2.6.1-9.el8.x86_64 2020-11-20T13:55:29.677 INFO:teuthology.orchestra.run.smithi040.stdout: libtool-2.4.6-25.el8.x86_64 2020-11-20T13:55:29.678 INFO:teuthology.orchestra.run.smithi040.stdout: elfutils-libelf-devel-0.176-5.el8.x86_64 2020-11-20T13:55:29.678 INFO:teuthology.orchestra.run.smithi040.stdout: libaio-devel-0.3.112-1.el8.x86_64 2020-11-20T13:55:29.679 INFO:teuthology.orchestra.run.smithi040.stdout: libuuid-devel-2.32.1-17.el8.x86_64 2020-11-20T13:55:29.679 INFO:teuthology.orchestra.run.smithi040.stdout: openssl-devel-1:1.1.1c-2.el8_1.1.x86_64 2020-11-20T13:55:29.679 INFO:teuthology.orchestra.run.smithi040.stdout: xfsprogs-devel-5.0.0-1.el8.x86_64 2020-11-20T13:55:29.679 INFO:teuthology.orchestra.run.smithi040.stdout: python3-html5lib-1:0.999999999-6.el8.noarch 2020-11-20T13:55:29.680 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cssselect-0.9.2-10.el8.noarch 2020-11-20T13:55:29.680 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-common-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:29.680 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mds-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:29.680 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mon-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:29.681 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-osd-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:29.681 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-selinux-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:29.681 INFO:teuthology.orchestra.run.smithi040.stdout: libradosstriper1-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:29.681 INFO:teuthology.orchestra.run.smithi040.stdout: librgw2-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:29.682 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-argparse-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:29.682 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-common-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:55:29.682 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-grafana-dashboards-2:15.2.5-588.ga7b62340.el8.noarch 2020-11-20T13:55:29.682 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-modules-core-2:15.2.5-588.ga7b62340.el8.noarch 2020-11-20T13:55:29.682 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-prometheus-alerts-2:15.2.5-588.ga7b62340.el8.noarch 2020-11-20T13:55:29.683 INFO:teuthology.orchestra.run.smithi040.stdout: automake-1.16.1-6.el8.noarch 2020-11-20T13:55:29.683 INFO:teuthology.orchestra.run.smithi040.stdout: jq-1.5-12.el8.x86_64 2020-11-20T13:55:29.683 INFO:teuthology.orchestra.run.smithi040.stdout: librdkafka-0.11.4-1.el8.x86_64 2020-11-20T13:55:29.683 INFO:teuthology.orchestra.run.smithi040.stdout: lttng-ust-2.8.1-11.el8.x86_64 2020-11-20T13:55:29.684 INFO:teuthology.orchestra.run.smithi040.stdout: oniguruma-6.8.2-1.el8.x86_64 2020-11-20T13:55:29.684 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-0.3.3-2.el8.x86_64 2020-11-20T13:55:29.684 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-threads-0.3.3-2.el8.x86_64 2020-11-20T13:55:29.684 INFO:teuthology.orchestra.run.smithi040.stdout: python3-babel-2.5.1-5.el8.noarch 2020-11-20T13:55:29.685 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2020-11-20T13:55:29.685 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpatch-1.21-2.el8.noarch 2020-11-20T13:55:29.685 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpointer-1.10-11.el8.noarch 2020-11-20T13:55:29.686 INFO:teuthology.orchestra.run.smithi040.stdout: python3-lxml-4.2.3-1.el8.x86_64 2020-11-20T13:55:29.686 INFO:teuthology.orchestra.run.smithi040.stdout: python3-mako-1.0.6-13.el8.noarch 2020-11-20T13:55:29.686 INFO:teuthology.orchestra.run.smithi040.stdout: python3-markupsafe-0.23-19.el8.x86_64 2020-11-20T13:55:29.687 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2020-11-20T13:55:29.687 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2020-11-20T13:55:29.687 INFO:teuthology.orchestra.run.smithi040.stdout: python3-prettytable-0.7.2-14.el8.noarch 2020-11-20T13:55:29.687 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2020-11-20T13:55:29.688 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2020-11-20T13:55:29.688 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pytz-2017.2-9.el8.noarch 2020-11-20T13:55:29.688 INFO:teuthology.orchestra.run.smithi040.stdout: python3-scipy-1.0.0-20.module_el8.1.0+245+c39af44f.x86_64 2020-11-20T13:55:29.688 INFO:teuthology.orchestra.run.smithi040.stdout: python3-webencodings-0.5.1-6.el8.noarch 2020-11-20T13:55:29.689 INFO:teuthology.orchestra.run.smithi040.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2020-11-20T13:55:29.690 INFO:teuthology.orchestra.run.smithi040.stdout: socat-1.7.3.2-6.el8.x86_64 2020-11-20T13:55:29.690 INFO:teuthology.orchestra.run.smithi040.stdout: fuse-2.9.7-12.el8.x86_64 2020-11-20T13:55:29.690 INFO:teuthology.orchestra.run.smithi040.stdout: fuse-common-3.2.1-12.el8.x86_64 2020-11-20T13:55:29.690 INFO:teuthology.orchestra.run.smithi040.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2020-11-20T13:55:29.691 INFO:teuthology.orchestra.run.smithi040.stdout: krb5-devel-1.17-9.el8.x86_64 2020-11-20T13:55:29.691 INFO:teuthology.orchestra.run.smithi040.stdout: libcom_err-devel-1.44.6-3.el8.x86_64 2020-11-20T13:55:29.691 INFO:teuthology.orchestra.run.smithi040.stdout: libgfortran-8.3.1-4.5.el8.x86_64 2020-11-20T13:55:29.692 INFO:teuthology.orchestra.run.smithi040.stdout: libquadmath-8.3.1-4.5.el8.x86_64 2020-11-20T13:55:29.692 INFO:teuthology.orchestra.run.smithi040.stdout: librabbitmq-0.9.0-1.el8.x86_64 2020-11-20T13:55:29.692 INFO:teuthology.orchestra.run.smithi040.stdout: libselinux-devel-2.9-2.1.el8.x86_64 2020-11-20T13:55:29.692 INFO:teuthology.orchestra.run.smithi040.stdout: libsepol-devel-2.9-1.el8.x86_64 2020-11-20T13:55:29.693 INFO:teuthology.orchestra.run.smithi040.stdout: libverto-devel-0.3.0-5.el8.x86_64 2020-11-20T13:55:29.693 INFO:teuthology.orchestra.run.smithi040.stdout: libxslt-1.1.32-3.el8.x86_64 2020-11-20T13:55:29.693 INFO:teuthology.orchestra.run.smithi040.stdout: pcre2-devel-10.32-1.el8.x86_64 2020-11-20T13:55:29.693 INFO:teuthology.orchestra.run.smithi040.stdout: pcre2-utf16-10.32-1.el8.x86_64 2020-11-20T13:55:29.694 INFO:teuthology.orchestra.run.smithi040.stdout: pcre2-utf32-10.32-1.el8.x86_64 2020-11-20T13:55:29.694 INFO:teuthology.orchestra.run.smithi040.stdout: python3-chardet-3.0.4-7.el8.noarch 2020-11-20T13:55:29.694 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jwt-1.6.1-2.el8.noarch 2020-11-20T13:55:29.694 INFO:teuthology.orchestra.run.smithi040.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2020-11-20T13:55:29.695 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pysocks-1.6.8-3.el8.noarch 2020-11-20T13:55:29.695 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-2.20.0-2.1.el8_1.noarch 2020-11-20T13:55:29.695 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2020-11-20T13:55:29.696 INFO:teuthology.orchestra.run.smithi040.stdout: python3-urllib3-1.24.2-2.el8.noarch 2020-11-20T13:55:29.696 INFO:teuthology.orchestra.run.smithi040.stdout: zlib-devel-1.2.11-10.el8.x86_64 2020-11-20T13:55:29.696 INFO:teuthology.orchestra.run.smithi040.stdout: fmt-6.2.1-1.el8.x86_64 2020-11-20T13:55:29.696 INFO:teuthology.orchestra.run.smithi040.stdout: gperftools-libs-2.7-6.el8.x86_64 2020-11-20T13:55:29.697 INFO:teuthology.orchestra.run.smithi040.stdout: leveldb-1.22-1.el8.x86_64 2020-11-20T13:55:29.697 INFO:teuthology.orchestra.run.smithi040.stdout: liboath-2.6.2-3.el8.x86_64 2020-11-20T13:55:29.697 INFO:teuthology.orchestra.run.smithi040.stdout: libunwind-1.3.1-3.el8.x86_64 2020-11-20T13:55:29.697 INFO:teuthology.orchestra.run.smithi040.stdout: python3-apipkg-1.5-6.el8.noarch 2020-11-20T13:55:29.698 INFO:teuthology.orchestra.run.smithi040.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2020-11-20T13:55:29.698 INFO:teuthology.orchestra.run.smithi040.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2020-11-20T13:55:29.698 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cachetools-3.1.1-4.el8.noarch 2020-11-20T13:55:29.699 INFO:teuthology.orchestra.run.smithi040.stdout: python3-certifi-2018.10.15-7.el8.noarch 2020-11-20T13:55:29.699 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cheroot-8.2.1-1.el8.noarch 2020-11-20T13:55:29.699 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2020-11-20T13:55:29.699 INFO:teuthology.orchestra.run.smithi040.stdout: python3-execnet-1.7.1-1.el8.noarch 2020-11-20T13:55:29.700 INFO:teuthology.orchestra.run.smithi040.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2020-11-20T13:55:29.700 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-6.2-6.el8.noarch 2020-11-20T13:55:29.700 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2020-11-20T13:55:29.700 INFO:teuthology.orchestra.run.smithi040.stdout: python3-kubernetes-11.0.0-2.el8.noarch 2020-11-20T13:55:29.700 INFO:teuthology.orchestra.run.smithi040.stdout: python3-logutils-0.3.5-11.el8.noarch 2020-11-20T13:55:29.701 INFO:teuthology.orchestra.run.smithi040.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2020-11-20T13:55:29.701 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pecan-1.3.2-9.el8.noarch 2020-11-20T13:55:29.701 INFO:teuthology.orchestra.run.smithi040.stdout: python3-portend-2.6-1.el8.noarch 2020-11-20T13:55:29.701 INFO:teuthology.orchestra.run.smithi040.stdout: python3-remoto-1.1.4-4.el8.noarch 2020-11-20T13:55:29.702 INFO:teuthology.orchestra.run.smithi040.stdout: python3-repoze-lru-0.7-6.el8.noarch 2020-11-20T13:55:29.702 INFO:teuthology.orchestra.run.smithi040.stdout: python3-routes-2.4.1-12.el8.noarch 2020-11-20T13:55:29.702 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rsa-3.4.2-15.el8.noarch 2020-11-20T13:55:29.702 INFO:teuthology.orchestra.run.smithi040.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2020-11-20T13:55:29.703 INFO:teuthology.orchestra.run.smithi040.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2020-11-20T13:55:29.703 INFO:teuthology.orchestra.run.smithi040.stdout: python3-tempora-1.14.1-5.el8.noarch 2020-11-20T13:55:29.703 INFO:teuthology.orchestra.run.smithi040.stdout: python3-trustme-0.6.0-4.el8.noarch 2020-11-20T13:55:29.703 INFO:teuthology.orchestra.run.smithi040.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2020-11-20T13:55:29.703 INFO:teuthology.orchestra.run.smithi040.stdout: python3-webob-1.8.5-1.el8.1.noarch 2020-11-20T13:55:29.704 INFO:teuthology.orchestra.run.smithi040.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2020-11-20T13:55:29.704 INFO:teuthology.orchestra.run.smithi040.stdout: python3-webtest-2.0.33-1.el8.noarch 2020-11-20T13:55:29.704 INFO:teuthology.orchestra.run.smithi040.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2020-11-20T13:55:29.704 INFO:teuthology.orchestra.run.smithi040.stdout: xmlstarlet-1.6.1-11.el8.x86_64 2020-11-20T13:55:29.705 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:55:29.706 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2020-11-20T13:55:29.839 DEBUG:teuthology.parallel:result is None 2020-11-20T13:55:29.839 INFO:teuthology.orchestra.run.smithi040:> sudo lsb_release -is 2020-11-20T13:55:29.894 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS 2020-11-20T13:55:29.896 DEBUG:teuthology.misc:System to be installed: CentOS 2020-11-20T13:55:29.896 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a7b62340ad27b26642bb54ec916a97c7e1f07a34 2020-11-20T13:55:30.070 INFO:teuthology.orchestra.run.smithi040:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2020-11-20T13:55:30.092 INFO:teuthology.orchestra.run.smithi040.stdout:15.2.5-588.ga7b62340.el8 2020-11-20T13:55:30.092 INFO:teuthology.packaging:The installed version of ceph is 15.2.5-588.ga7b62340.el8 2020-11-20T13:55:30.093 INFO:teuthology.task.install:The correct ceph version 15.2.5-588.ga7b62340 is installed. 2020-11-20T13:55:30.094 INFO:teuthology.orchestra.run.smithi116:> sudo lsb_release -is 2020-11-20T13:55:30.148 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS 2020-11-20T13:55:30.149 DEBUG:teuthology.misc:System to be installed: CentOS 2020-11-20T13:55:30.150 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a7b62340ad27b26642bb54ec916a97c7e1f07a34 2020-11-20T13:55:30.316 INFO:teuthology.orchestra.run.smithi116:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2020-11-20T13:55:30.337 INFO:teuthology.orchestra.run.smithi116.stdout:15.2.5-588.ga7b62340.el8 2020-11-20T13:55:30.338 INFO:teuthology.packaging:The installed version of ceph is 15.2.5-588.ga7b62340.el8 2020-11-20T13:55:30.338 INFO:teuthology.task.install:The correct ceph version 15.2.5-588.ga7b62340 is installed. 2020-11-20T13:55:30.339 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2020-11-20T13:55:30.339 INFO:teuthology.orchestra.run.smithi040:> set -ex 2020-11-20T13:55:30.340 INFO:teuthology.orchestra.run.smithi040:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-20T13:55:30.354 INFO:teuthology.orchestra.run.smithi040.stderr:+ sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-20T13:55:30.373 INFO:teuthology.orchestra.run.smithi040.stderr:24+1 records in 2020-11-20T13:55:30.373 INFO:teuthology.orchestra.run.smithi040.stderr:24+1 records out 2020-11-20T13:55:30.373 INFO:teuthology.orchestra.run.smithi040.stderr:12667 bytes (13 kB, 12 KiB) copied, 0.000110763 s, 114 MB/s 2020-11-20T13:55:30.375 INFO:teuthology.orchestra.run.smithi116:> set -ex 2020-11-20T13:55:30.375 INFO:teuthology.orchestra.run.smithi116:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-20T13:55:30.393 INFO:teuthology.orchestra.run.smithi116.stderr:+ sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-20T13:55:30.408 INFO:teuthology.orchestra.run.smithi116.stderr:24+1 records in 2020-11-20T13:55:30.408 INFO:teuthology.orchestra.run.smithi116.stderr:24+1 records out 2020-11-20T13:55:30.408 INFO:teuthology.orchestra.run.smithi116.stderr:12667 bytes (13 kB, 12 KiB) copied, 7.7698e-05 s, 163 MB/s 2020-11-20T13:55:30.409 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2020-11-20T13:55:30.410 INFO:teuthology.orchestra.run.smithi040:> set -ex 2020-11-20T13:55:30.410 INFO:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/daemon-helper 2020-11-20T13:55:30.427 INFO:teuthology.orchestra.run.smithi040.stderr:+ sudo dd of=/usr/bin/daemon-helper 2020-11-20T13:55:30.444 INFO:teuthology.orchestra.run.smithi040.stderr:5+1 records in 2020-11-20T13:55:30.445 INFO:teuthology.orchestra.run.smithi040.stderr:5+1 records out 2020-11-20T13:55:30.445 INFO:teuthology.orchestra.run.smithi040.stderr:3058 bytes (3.1 kB, 3.0 KiB) copied, 4.3217e-05 s, 70.8 MB/s 2020-11-20T13:55:30.446 INFO:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-11-20T13:55:30.518 INFO:teuthology.orchestra.run.smithi116:> set -ex 2020-11-20T13:55:30.519 INFO:teuthology.orchestra.run.smithi116:> sudo dd of=/usr/bin/daemon-helper 2020-11-20T13:55:30.535 INFO:teuthology.orchestra.run.smithi116.stderr:+ sudo dd of=/usr/bin/daemon-helper 2020-11-20T13:55:30.550 INFO:teuthology.orchestra.run.smithi116.stderr:5+1 records in 2020-11-20T13:55:30.550 INFO:teuthology.orchestra.run.smithi116.stderr:5+1 records out 2020-11-20T13:55:30.550 INFO:teuthology.orchestra.run.smithi116.stderr:3058 bytes (3.1 kB, 3.0 KiB) copied, 4.1558e-05 s, 73.6 MB/s 2020-11-20T13:55:30.551 INFO:teuthology.orchestra.run.smithi116:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-11-20T13:55:30.620 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2020-11-20T13:55:30.621 INFO:teuthology.orchestra.run.smithi040:> set -ex 2020-11-20T13:55:30.621 INFO:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/adjust-ulimits 2020-11-20T13:55:30.635 INFO:teuthology.orchestra.run.smithi040.stderr:+ sudo dd of=/usr/bin/adjust-ulimits 2020-11-20T13:55:30.651 INFO:teuthology.orchestra.run.smithi040.stderr:0+1 records in 2020-11-20T13:55:30.652 INFO:teuthology.orchestra.run.smithi040.stderr:0+1 records out 2020-11-20T13:55:30.652 INFO:teuthology.orchestra.run.smithi040.stderr:299 bytes copied, 2.907e-05 s, 10.3 MB/s 2020-11-20T13:55:30.653 INFO:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-11-20T13:55:30.723 INFO:teuthology.orchestra.run.smithi116:> set -ex 2020-11-20T13:55:30.723 INFO:teuthology.orchestra.run.smithi116:> sudo dd of=/usr/bin/adjust-ulimits 2020-11-20T13:55:30.738 INFO:teuthology.orchestra.run.smithi116.stderr:+ sudo dd of=/usr/bin/adjust-ulimits 2020-11-20T13:55:30.751 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records in 2020-11-20T13:55:30.752 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records out 2020-11-20T13:55:30.752 INFO:teuthology.orchestra.run.smithi116.stderr:299 bytes copied, 2.9534e-05 s, 10.1 MB/s 2020-11-20T13:55:30.753 INFO:teuthology.orchestra.run.smithi116:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-11-20T13:55:30.821 INFO:teuthology.run_tasks:Running task ceph... 2020-11-20T13:55:30.876 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2020-11-20T13:55:30.877 INFO:teuthology.orchestra.run.smithi040:> sudo chmod 777 /var/log/ceph 2020-11-20T13:55:30.880 INFO:teuthology.orchestra.run.smithi116:> sudo chmod 777 /var/log/ceph 2020-11-20T13:55:30.910 INFO:tasks.ceph:Disabling ceph logrotate... 2020-11-20T13:55:30.911 INFO:teuthology.orchestra.run.smithi040:> sudo rm -f -- /etc/logrotate.d/ceph 2020-11-20T13:55:30.952 INFO:teuthology.orchestra.run.smithi116:> sudo rm -f -- /etc/logrotate.d/ceph 2020-11-20T13:55:30.981 INFO:tasks.ceph:Creating extra log directories... 2020-11-20T13:55:30.981 INFO:teuthology.orchestra.run.smithi040:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2020-11-20T13:55:31.023 INFO:teuthology.orchestra.run.smithi116:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2020-11-20T13:55:31.052 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2020-11-20T13:55:31.053 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2020-11-20T13:55:31.053 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2020-11-20T13:55:31.053 INFO:teuthology.orchestra.run.smithi040:> set -ex 2020-11-20T13:55:31.054 INFO:teuthology.orchestra.run.smithi040:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-11-20T13:55:31.105 INFO:teuthology.orchestra.run.smithi040.stderr:+ dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-11-20T13:55:31.107 INFO:teuthology.orchestra.run.smithi040.stderr:0+1 records in 2020-11-20T13:55:31.107 INFO:teuthology.orchestra.run.smithi040.stderr:0+1 records out 2020-11-20T13:55:31.107 INFO:teuthology.orchestra.run.smithi040.stderr:408 bytes copied, 3.7576e-05 s, 10.9 MB/s 2020-11-20T13:55:31.108 INFO:teuthology.orchestra.run.smithi040:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2020-11-20T13:55:31.229 INFO:teuthology.orchestra.run.smithi040:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2020-11-20T13:55:31.260 INFO:teuthology.orchestra.run.smithi116:> set -ex 2020-11-20T13:55:31.261 INFO:teuthology.orchestra.run.smithi116:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-11-20T13:55:31.275 INFO:teuthology.orchestra.run.smithi116.stderr:+ dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-11-20T13:55:31.277 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records in 2020-11-20T13:55:31.277 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records out 2020-11-20T13:55:31.277 INFO:teuthology.orchestra.run.smithi116.stderr:408 bytes copied, 2.7405e-05 s, 14.9 MB/s 2020-11-20T13:55:31.277 INFO:teuthology.orchestra.run.smithi116:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2020-11-20T13:55:31.392 INFO:teuthology.orchestra.run.smithi116:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2020-11-20T13:55:31.419 INFO:tasks.ceph:Creating ceph cluster ceph... 2020-11-20T13:55:31.420 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2020-11-20T13:55:31.420 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2020-11-16_04:10:02-fs-octopus-distro-basic-smithi/5628507', 'branch': 'octopus', 'description': 'fs/basic_functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} mount/fuse objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} supported-random-distros$/{centos_latest} tasks/cap-flush}', 'email': 'ceph-qa@ceph.io', 'first_in_suite': False, 'job_id': '5628507', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'teuthology-2020-11-16_04:10:02-fs-octopus-distro-basic-smithi', 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 30}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '8.1', 'overrides': {'admin_socket': {'branch': 'octopus'}, 'ceph': {'cephfs_ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'Replacing daemon mds.a'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': 'a7b62340ad27b26642bb54ec916a97c7e1f07a34'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'sha1': 'a7b62340ad27b26642bb54ec916a97c7e1f07a34'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'octopus', 'sha1': 'a7b62340ad27b26642bb54ec916a97c7e1f07a34'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 1000, 'repo': 'git://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.1'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.2', 'client.3']], 'sha1': 'a7b62340ad27b26642bb54ec916a97c7e1f07a34', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'octopus', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_octopus/qa', 'suite_relpath': 'qa', 'suite_repo': 'git://git.ceph.com/ceph.git', 'suite_sha1': 'a7b62340ad27b26642bb54ec916a97c7e1f07a34', 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.lock_machines': (2, 'smithi')}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'sha1': 'a7b62340ad27b26642bb54ec916a97c7e1f07a34'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_cap_flush']}}], 'teuthology_branch': 'master', 'timestamp': '2020-11-16_04:10:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/worker.smithi.3621', 'targets': {'smithi040.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCc+NNCop57dEEIxzisCShx+4lQwqNnkwsklSPNn3QMj4SM0iuHLzR9gJES8K+idQbyVjnJdxNXsW9oqjMIGkAc61AWlcbAGZbhHLA93FICp9JjaONyJ10GHTiIzlCieksFsIJrzHEuBRoTOxzfmAoHlkM4QKp03eovsiornj4Y86di2Puk4H68QFw1q5HzZOJajU9jw7FNGABe2+2YNdd6Y/wAwh3rmpY43B4o1FrrLow6bpia+5Fe8/RN/fG30uEWZ0/6KP9C+QjLqnxyE4wnPx9fJXvXb3v4/oF+wfEKIsaW8/S5ufJm4x2/XxrTKSJF8DvNs3VNfZJ/ZZ0Q1/T5MJm0yv6Qqt+SO+ia6d5O2597FYiBPxLF8tVZpXWIdvYm+i8rxevgrdCN40FgYuukDuUrNEuaezpPXHKqTiQRu2vBXRfNd23F3B6wOKlpbJ+AwIOxuuSsAzRApymSMgwF22gKxqZvlflH600AUHdcuyTh2P3I7QEnHWJah5AcYts=', 'smithi116.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCuDX2LoD2ExaYCl+93xaiz+YKTfMJuUpuOuZUYBxtXw+Gd2Fc98v2x+Q/9Wyw8sE9yrIA2lJFZFo0QDr4/s+Vdil+vYqGsvqh9/TWzKJ3BCSSjNsO6Ux6oxj1PIqwHwWm8VUMLtw4E5nBlVlGwS1OsNNs8HSMKc1BWq8+CZj5ocEzy99vjhddvLLzHZliEyQLJVtaFGnLpMnpixCeRRgydkOiv2lbDUaBoeNyV3GcD3syouvI6hR9QaZtS46HY4QHVfvWwyvLITHvnn0lQkLhwWuhjndZ+Ly+jXUCWOF+Nv4COUj3OgX8G5eExRb97jE05DE1E8N2iX31HOH3UKIXZJ4yS3v4odJWd6kT8WYY16tVAZslvhYjZbOfoyISPHLPmJCAamgWo4hd01KhTDawrwsu0rvoe7HcL/IQqCbsaA5EpqWHYGQ5dptLlqfud5+FjlYkPnUsw55YWfZPCbb91nXhK7SWqdyHbC0zUtckqp/Lb5wSJ40AvpIuE589JMQE='}} 2020-11-20T13:55:31.421 INFO:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2020-11-20T13:55:31.423 INFO:teuthology.orchestra.run.smithi116:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2020-11-20T13:55:31.473 INFO:teuthology.orchestra.run.smithi040:> sudo install -d -m0777 -- /var/run/ceph 2020-11-20T13:55:31.478 INFO:teuthology.orchestra.run.smithi116:> sudo install -d -m0777 -- /var/run/ceph 2020-11-20T13:55:31.540 INFO:teuthology.orchestra.run.smithi040:> set -ex 2020-11-20T13:55:31.540 INFO:teuthology.orchestra.run.smithi040:> dd if=/scratch_devs of=/dev/stdout 2020-11-20T13:55:31.559 INFO:teuthology.orchestra.run.smithi040.stderr:+ dd if=/scratch_devs of=/dev/stdout 2020-11-20T13:55:31.561 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/vg_nvme/lv_1 2020-11-20T13:55:31.561 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/vg_nvme/lv_2 2020-11-20T13:55:31.561 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/vg_nvme/lv_3 2020-11-20T13:55:31.561 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/vg_nvme/lv_4 2020-11-20T13:55:31.562 INFO:teuthology.orchestra.run.smithi040.stderr:0+1 records in 2020-11-20T13:55:31.562 INFO:teuthology.orchestra.run.smithi040.stderr:0+1 records out 2020-11-20T13:55:31.562 INFO:teuthology.orchestra.run.smithi040.stderr:72 bytes copied, 3.4418e-05 s, 2.1 MB/s 2020-11-20T13:55:31.562 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2020-11-20T13:55:31.563 INFO:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2020-11-20T13:55:31.615 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2020-11-20T13:55:31.615 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-20T13:55:31.616 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 6h/6d Inode: 63683 Links: 1 2020-11-20T13:55:31.616 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-20T13:55:31.616 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2020-11-20T13:55:31.616 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2020-11-20 13:55:27.502880863 +0000 2020-11-20T13:55:31.617 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2020-11-20 13:53:32.724645375 +0000 2020-11-20T13:55:31.617 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2020-11-20 13:53:32.724645375 +0000 2020-11-20T13:55:31.617 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: - 2020-11-20T13:55:31.632 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2020-11-20T13:55:31.632 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2020-11-20T13:55:31.632 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000210415 s, 2.4 MB/s 2020-11-20T13:55:31.634 INFO:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2020-11-20T13:55:31.689 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2020-11-20T13:55:31.690 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-20T13:55:31.690 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 6h/6d Inode: 56278 Links: 1 2020-11-20T13:55:31.691 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-20T13:55:31.691 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2020-11-20T13:55:31.691 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2020-11-20 13:55:27.502880863 +0000 2020-11-20T13:55:31.692 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2020-11-20 13:53:33.065644254 +0000 2020-11-20T13:55:31.692 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2020-11-20 13:53:33.065644254 +0000 2020-11-20T13:55:31.693 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: - 2020-11-20T13:55:31.706 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2020-11-20T13:55:31.706 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2020-11-20T13:55:31.707 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000233423 s, 2.2 MB/s 2020-11-20T13:55:31.708 INFO:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2020-11-20T13:55:31.763 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2020-11-20T13:55:31.763 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-20T13:55:31.763 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 6h/6d Inode: 63791 Links: 1 2020-11-20T13:55:31.763 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-20T13:55:31.764 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2020-11-20T13:55:31.764 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2020-11-20 13:55:27.502880863 +0000 2020-11-20T13:55:31.764 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2020-11-20 13:53:33.410643119 +0000 2020-11-20T13:55:31.764 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2020-11-20 13:53:33.410643119 +0000 2020-11-20T13:55:31.764 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: - 2020-11-20T13:55:31.779 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2020-11-20T13:55:31.780 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2020-11-20T13:55:31.780 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000218974 s, 2.3 MB/s 2020-11-20T13:55:31.782 INFO:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2020-11-20T13:55:31.836 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2020-11-20T13:55:31.837 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-20T13:55:31.837 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 6h/6d Inode: 62572 Links: 1 2020-11-20T13:55:31.837 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-20T13:55:31.838 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2020-11-20T13:55:31.838 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2020-11-20 13:55:27.503880850 +0000 2020-11-20T13:55:31.838 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2020-11-20 13:53:33.749642004 +0000 2020-11-20T13:55:31.838 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2020-11-20 13:53:33.749642004 +0000 2020-11-20T13:55:31.839 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: - 2020-11-20T13:55:31.853 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2020-11-20T13:55:31.853 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2020-11-20T13:55:31.853 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.00017553 s, 2.9 MB/s 2020-11-20T13:55:31.856 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-11-20T13:55:31.856 INFO:teuthology.orchestra.run.smithi116:> set -ex 2020-11-20T13:55:31.857 INFO:teuthology.orchestra.run.smithi116:> dd if=/scratch_devs of=/dev/stdout 2020-11-20T13:55:31.870 INFO:teuthology.orchestra.run.smithi116.stderr:+ dd if=/scratch_devs of=/dev/stdout 2020-11-20T13:55:31.871 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/vg_nvme/lv_1 2020-11-20T13:55:31.871 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/vg_nvme/lv_2 2020-11-20T13:55:31.872 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/vg_nvme/lv_3 2020-11-20T13:55:31.872 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/vg_nvme/lv_4 2020-11-20T13:55:31.872 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records in 2020-11-20T13:55:31.872 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records out 2020-11-20T13:55:31.873 INFO:teuthology.orchestra.run.smithi116.stderr:72 bytes copied, 3.7694e-05 s, 1.9 MB/s 2020-11-20T13:55:31.873 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2020-11-20T13:55:31.873 INFO:teuthology.orchestra.run.smithi116:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2020-11-20T13:55:31.926 INFO:teuthology.orchestra.run.smithi116.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2020-11-20T13:55:31.927 INFO:teuthology.orchestra.run.smithi116.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-20T13:55:31.927 INFO:teuthology.orchestra.run.smithi116.stdout:Device: 6h/6d Inode: 61282 Links: 1 2020-11-20T13:55:31.927 INFO:teuthology.orchestra.run.smithi116.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-20T13:55:31.927 INFO:teuthology.orchestra.run.smithi116.stdout:Context: system_u:object_r:device_t:s0 2020-11-20T13:55:31.927 INFO:teuthology.orchestra.run.smithi116.stdout:Access: 2020-11-20 13:55:19.706809534 +0000 2020-11-20T13:55:31.928 INFO:teuthology.orchestra.run.smithi116.stdout:Modify: 2020-11-20 13:53:30.400479730 +0000 2020-11-20T13:55:31.928 INFO:teuthology.orchestra.run.smithi116.stdout:Change: 2020-11-20 13:53:30.400479730 +0000 2020-11-20T13:55:31.928 INFO:teuthology.orchestra.run.smithi116.stdout: Birth: - 2020-11-20T13:55:31.940 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records in 2020-11-20T13:55:31.941 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records out 2020-11-20T13:55:31.941 INFO:teuthology.orchestra.run.smithi116.stderr:512 bytes copied, 0.000246173 s, 2.1 MB/s 2020-11-20T13:55:31.944 INFO:teuthology.orchestra.run.smithi116:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2020-11-20T13:55:31.999 INFO:teuthology.orchestra.run.smithi116.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2020-11-20T13:55:32.000 INFO:teuthology.orchestra.run.smithi116.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-20T13:55:32.000 INFO:teuthology.orchestra.run.smithi116.stdout:Device: 6h/6d Inode: 62577 Links: 1 2020-11-20T13:55:32.000 INFO:teuthology.orchestra.run.smithi116.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-20T13:55:32.000 INFO:teuthology.orchestra.run.smithi116.stdout:Context: system_u:object_r:device_t:s0 2020-11-20T13:55:32.001 INFO:teuthology.orchestra.run.smithi116.stdout:Access: 2020-11-20 13:55:19.706809534 +0000 2020-11-20T13:55:32.001 INFO:teuthology.orchestra.run.smithi116.stdout:Modify: 2020-11-20 13:53:30.718474566 +0000 2020-11-20T13:55:32.001 INFO:teuthology.orchestra.run.smithi116.stdout:Change: 2020-11-20 13:53:30.718474566 +0000 2020-11-20T13:55:32.002 INFO:teuthology.orchestra.run.smithi116.stdout: Birth: - 2020-11-20T13:55:32.013 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records in 2020-11-20T13:55:32.013 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records out 2020-11-20T13:55:32.013 INFO:teuthology.orchestra.run.smithi116.stderr:512 bytes copied, 0.000211854 s, 2.4 MB/s 2020-11-20T13:55:32.015 INFO:teuthology.orchestra.run.smithi116:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2020-11-20T13:55:32.070 INFO:teuthology.orchestra.run.smithi116.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2020-11-20T13:55:32.070 INFO:teuthology.orchestra.run.smithi116.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-20T13:55:32.071 INFO:teuthology.orchestra.run.smithi116.stdout:Device: 6h/6d Inode: 59277 Links: 1 2020-11-20T13:55:32.071 INFO:teuthology.orchestra.run.smithi116.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-20T13:55:32.071 INFO:teuthology.orchestra.run.smithi116.stdout:Context: system_u:object_r:device_t:s0 2020-11-20T13:55:32.071 INFO:teuthology.orchestra.run.smithi116.stdout:Access: 2020-11-20 13:55:19.706809534 +0000 2020-11-20T13:55:32.071 INFO:teuthology.orchestra.run.smithi116.stdout:Modify: 2020-11-20 13:53:31.050469176 +0000 2020-11-20T13:55:32.072 INFO:teuthology.orchestra.run.smithi116.stdout:Change: 2020-11-20 13:53:31.050469176 +0000 2020-11-20T13:55:32.072 INFO:teuthology.orchestra.run.smithi116.stdout: Birth: - 2020-11-20T13:55:32.083 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records in 2020-11-20T13:55:32.083 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records out 2020-11-20T13:55:32.084 INFO:teuthology.orchestra.run.smithi116.stderr:512 bytes copied, 0.000203493 s, 2.5 MB/s 2020-11-20T13:55:32.086 INFO:teuthology.orchestra.run.smithi116:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2020-11-20T13:55:32.141 INFO:teuthology.orchestra.run.smithi116.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2020-11-20T13:55:32.141 INFO:teuthology.orchestra.run.smithi116.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-20T13:55:32.141 INFO:teuthology.orchestra.run.smithi116.stdout:Device: 6h/6d Inode: 61898 Links: 1 2020-11-20T13:55:32.142 INFO:teuthology.orchestra.run.smithi116.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-20T13:55:32.142 INFO:teuthology.orchestra.run.smithi116.stdout:Context: system_u:object_r:device_t:s0 2020-11-20T13:55:32.142 INFO:teuthology.orchestra.run.smithi116.stdout:Access: 2020-11-20 13:55:19.706809534 +0000 2020-11-20T13:55:32.142 INFO:teuthology.orchestra.run.smithi116.stdout:Modify: 2020-11-20 13:53:31.375463899 +0000 2020-11-20T13:55:32.142 INFO:teuthology.orchestra.run.smithi116.stdout:Change: 2020-11-20 13:53:31.375463899 +0000 2020-11-20T13:55:32.143 INFO:teuthology.orchestra.run.smithi116.stdout: Birth: - 2020-11-20T13:55:32.154 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records in 2020-11-20T13:55:32.155 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records out 2020-11-20T13:55:32.155 INFO:teuthology.orchestra.run.smithi116.stderr:512 bytes copied, 0.000200746 s, 2.6 MB/s 2020-11-20T13:55:32.157 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-11-20T13:55:32.157 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi040.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi116.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2020-11-20T13:55:32.158 INFO:tasks.ceph:Generating config... 2020-11-20T13:55:32.159 INFO:tasks.ceph:[client] client mount timeout = 600 2020-11-20T13:55:32.160 INFO:tasks.ceph:[client] debug client = 20 2020-11-20T13:55:32.160 INFO:tasks.ceph:[client] debug ms = 1 2020-11-20T13:55:32.160 INFO:tasks.ceph:[client] pid file = 2020-11-20T13:55:32.161 INFO:tasks.ceph:[client] rados mon op timeout = 15m 2020-11-20T13:55:32.161 INFO:tasks.ceph:[client] rados osd op timeout = 15m 2020-11-20T13:55:32.161 INFO:tasks.ceph:[mds] debug mds = 20 2020-11-20T13:55:32.162 INFO:tasks.ceph:[mds] debug ms = 1 2020-11-20T13:55:32.162 INFO:tasks.ceph:[mds] mds bal frag = True 2020-11-20T13:55:32.163 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2020-11-20T13:55:32.164 INFO:tasks.ceph:[mds] mds bal merge size = 5 2020-11-20T13:55:32.164 INFO:tasks.ceph:[mds] mds bal split bits = 3 2020-11-20T13:55:32.165 INFO:tasks.ceph:[mds] mds bal split size = 100 2020-11-20T13:55:32.165 INFO:tasks.ceph:[mds] mds debug frag = True 2020-11-20T13:55:32.165 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2020-11-20T13:55:32.166 INFO:tasks.ceph:[mds] mds op complaint time = 180 2020-11-20T13:55:32.166 INFO:tasks.ceph:[mds] mds verify scatter = True 2020-11-20T13:55:32.166 INFO:tasks.ceph:[mds] osd op complaint time = 180 2020-11-20T13:55:32.166 INFO:tasks.ceph:[mds] rados mon op timeout = 15m 2020-11-20T13:55:32.167 INFO:tasks.ceph:[mds] rados osd op timeout = 15m 2020-11-20T13:55:32.167 INFO:tasks.ceph:[mgr] debug mgr = 20 2020-11-20T13:55:32.167 INFO:tasks.ceph:[mgr] debug ms = 1 2020-11-20T13:55:32.168 INFO:tasks.ceph:[mon] debug mon = 20 2020-11-20T13:55:32.168 INFO:tasks.ceph:[mon] debug ms = 1 2020-11-20T13:55:32.168 INFO:tasks.ceph:[mon] debug paxos = 20 2020-11-20T13:55:32.168 INFO:tasks.ceph:[mon] mon op complaint time = 120 2020-11-20T13:55:32.169 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2020-11-20T13:55:32.169 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2020-11-20T13:55:32.169 INFO:tasks.ceph:[osd] debug bluefs = 20 2020-11-20T13:55:32.170 INFO:tasks.ceph:[osd] debug bluestore = 20 2020-11-20T13:55:32.170 INFO:tasks.ceph:[osd] debug filestore = 20 2020-11-20T13:55:32.170 INFO:tasks.ceph:[osd] debug journal = 20 2020-11-20T13:55:32.170 INFO:tasks.ceph:[osd] debug ms = 20 2020-11-20T13:55:32.171 INFO:tasks.ceph:[osd] debug osd = 25 2020-11-20T13:55:32.171 INFO:tasks.ceph:[osd] debug rocksdb = 10 2020-11-20T13:55:32.173 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2020-11-20T13:55:32.173 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2020-11-20T13:55:32.173 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2020-11-20T13:55:32.174 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2020-11-20T13:55:32.174 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2020-11-20T13:55:32.174 INFO:tasks.ceph:[osd] osd op complaint time = 180 2020-11-20T13:55:32.175 INFO:tasks.ceph:Setting up mon.a... 2020-11-20T13:55:32.175 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2020-11-20T13:55:32.244 INFO:teuthology.orchestra.run.smithi040.stdout:creating /etc/ceph/ceph.keyring 2020-11-20T13:55:32.246 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2020-11-20T13:55:32.293 INFO:teuthology.orchestra.run.smithi040:> sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-20T13:55:32.324 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.40'), ('mon.b', '172.21.15.116'), ('mon.c', '[v2:172.21.15.116:3301,v1:172.21.15.116:6790]')] 2020-11-20T13:55:32.325 INFO:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool --create --clobber --enable-all-features --add a 172.21.15.40 --add b 172.21.15.116 --addv c '[v2:172.21.15.116:3301,v1:172.21.15.116:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2020-11-20T13:55:32.402 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2020-11-20T13:55:32.403 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool: generated fsid da1ed737-5174-430f-a3c9-56f6bbfad13c 2020-11-20T13:55:32.403 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 0 2020-11-20T13:55:32.403 INFO:teuthology.orchestra.run.smithi040.stdout:fsid da1ed737-5174-430f-a3c9-56f6bbfad13c 2020-11-20T13:55:32.404 INFO:teuthology.orchestra.run.smithi040.stdout:last_changed 2020-11-20T13:55:32.403251+0000 2020-11-20T13:55:32.404 INFO:teuthology.orchestra.run.smithi040.stdout:created 2020-11-20T13:55:32.403251+0000 2020-11-20T13:55:32.404 INFO:teuthology.orchestra.run.smithi040.stdout:min_mon_release 0 (unknown) 2020-11-20T13:55:32.404 INFO:teuthology.orchestra.run.smithi040.stdout:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2020-11-20T13:55:32.404 INFO:teuthology.orchestra.run.smithi040.stdout:1: [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] mon.b 2020-11-20T13:55:32.405 INFO:teuthology.orchestra.run.smithi040.stdout:2: [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] mon.c 2020-11-20T13:55:32.405 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2020-11-20T13:55:32.406 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID da1ed737-5174-430f-a3c9-56f6bbfad13c... 2020-11-20T13:55:32.407 INFO:teuthology.orchestra.run.smithi040:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T13:55:32.448 INFO:teuthology.orchestra.run.smithi116:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T13:55:32.506 INFO:teuthology.orchestra.run.smithi116.stdout:[global] 2020-11-20T13:55:32.507 INFO:teuthology.orchestra.run.smithi116.stdout: chdir = "" 2020-11-20T13:55:32.507 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T13:55:32.507 INFO:teuthology.orchestra.run.smithi116.stdout: auth supported = cephx 2020-11-20T13:55:32.508 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T13:55:32.508 INFO:teuthology.orchestra.run.smithi116.stdout: filestore xattr use omap = true 2020-11-20T13:55:32.508 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T13:55:32.508 INFO:teuthology.orchestra.run.smithi116.stdout: mon clock drift allowed = 1.000 2020-11-20T13:55:32.508 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T13:55:32.509 INFO:teuthology.orchestra.run.smithi116.stdout: osd crush chooseleaf type = 0 2020-11-20T13:55:32.509 INFO:teuthology.orchestra.run.smithi116.stdout: auth debug = true 2020-11-20T13:55:32.509 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T13:55:32.509 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on old message = true 2020-11-20T13:55:32.510 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on bug = true 2020-11-20T13:55:32.510 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T13:55:32.510 INFO:teuthology.orchestra.run.smithi116.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T13:55:32.510 INFO:teuthology.orchestra.run.smithi116.stdout: mon pg warn max object skew = 0 2020-11-20T13:55:32.511 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T13:55:32.511 INFO:teuthology.orchestra.run.smithi116.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T13:55:32.511 INFO:teuthology.orchestra.run.smithi116.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T13:55:32.511 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T13:55:32.512 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default size = 2 2020-11-20T13:55:32.512 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T13:55:32.512 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow primary affinity = true 2020-11-20T13:55:32.512 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow pg remap = true 2020-11-20T13:55:32.512 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on legacy crush tunables = false 2020-11-20T13:55:32.513 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on crush straw calc version zero = false 2020-11-20T13:55:32.513 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on no sortbitwise = false 2020-11-20T13:55:32.513 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on osd down out interval zero = false 2020-11-20T13:55:32.513 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on too few osds = false 2020-11-20T13:55:32.514 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T13:55:32.514 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T13:55:32.514 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T13:55:32.514 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T13:55:32.515 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T13:55:32.515 INFO:teuthology.orchestra.run.smithi116.stdout: osd default data pool replay window = 5 2020-11-20T13:55:32.516 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T13:55:32.516 INFO:teuthology.orchestra.run.smithi116.stdout: mon allow pool delete = true 2020-11-20T13:55:32.516 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T13:55:32.516 INFO:teuthology.orchestra.run.smithi116.stdout: mon cluster log file level = debug 2020-11-20T13:55:32.517 INFO:teuthology.orchestra.run.smithi116.stdout: debug asserts on shutdown = true 2020-11-20T13:55:32.517 INFO:teuthology.orchestra.run.smithi116.stdout: mon host = "172.21.15.40,172.21.15.116,[v2:172.21.15.116:3301,v1:172.21.15.116:6790]" 2020-11-20T13:55:32.517 INFO:teuthology.orchestra.run.smithi116.stdout: fsid = da1ed737-5174-430f-a3c9-56f6bbfad13c 2020-11-20T13:55:32.517 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T13:55:32.517 INFO:teuthology.orchestra.run.smithi116.stdout:[osd] 2020-11-20T13:55:32.520 INFO:teuthology.orchestra.run.smithi116.stdout: osd journal size = 100 2020-11-20T13:55:32.520 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T13:55:32.520 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub load threshold = 5.0 2020-11-20T13:55:32.520 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub max interval = 600 2020-11-20T13:55:32.521 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T13:55:32.521 INFO:teuthology.orchestra.run.smithi116.stdout: osd recover clone overlap = true 2020-11-20T13:55:32.521 INFO:teuthology.orchestra.run.smithi116.stdout: osd recovery max chunk = 1048576 2020-11-20T13:55:32.521 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T13:55:32.521 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug shutdown = true 2020-11-20T13:55:32.522 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug op order = true 2020-11-20T13:55:32.522 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug verify stray on activate = true 2020-11-20T13:55:32.522 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T13:55:32.522 INFO:teuthology.orchestra.run.smithi116.stdout: osd open classes on start = true 2020-11-20T13:55:32.522 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug pg log writeout = true 2020-11-20T13:55:32.523 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T13:55:32.523 INFO:teuthology.orchestra.run.smithi116.stdout: osd deep scrub update digest min age = 30 2020-11-20T13:55:32.523 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T13:55:32.523 INFO:teuthology.orchestra.run.smithi116.stdout: osd map max advance = 10 2020-11-20T13:55:32.523 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T13:55:32.524 INFO:teuthology.orchestra.run.smithi116.stdout: journal zero on create = true 2020-11-20T13:55:32.524 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T13:55:32.524 INFO:teuthology.orchestra.run.smithi116.stdout: filestore ondisk finisher threads = 3 2020-11-20T13:55:32.524 INFO:teuthology.orchestra.run.smithi116.stdout: filestore apply finisher threads = 3 2020-11-20T13:55:32.524 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T13:55:32.525 INFO:teuthology.orchestra.run.smithi116.stdout: bdev debug aio = true 2020-11-20T13:55:32.525 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug misdirected ops = true 2020-11-20T13:55:32.525 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore block size = 96636764160 2020-11-20T13:55:32.525 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore fsck on mount = True 2020-11-20T13:55:32.526 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluefs = 20 2020-11-20T13:55:32.526 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluestore = 20 2020-11-20T13:55:32.526 INFO:teuthology.orchestra.run.smithi116.stdout: debug filestore = 20 2020-11-20T13:55:32.526 INFO:teuthology.orchestra.run.smithi116.stdout: debug journal = 20 2020-11-20T13:55:32.526 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 20 2020-11-20T13:55:32.527 INFO:teuthology.orchestra.run.smithi116.stdout: debug osd = 25 2020-11-20T13:55:32.527 INFO:teuthology.orchestra.run.smithi116.stdout: debug rocksdb = 10 2020-11-20T13:55:32.528 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T13:55:32.528 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd full ratio = 0.9 2020-11-20T13:55:32.528 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd nearfull ratio = 0.8 2020-11-20T13:55:32.529 INFO:teuthology.orchestra.run.smithi116.stdout: osd failsafe full ratio = 0.95 2020-11-20T13:55:32.529 INFO:teuthology.orchestra.run.smithi116.stdout: osd objectstore = bluestore 2020-11-20T13:55:32.530 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T13:55:32.530 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T13:55:32.530 INFO:teuthology.orchestra.run.smithi116.stdout:[mgr] 2020-11-20T13:55:32.530 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T13:55:32.531 INFO:teuthology.orchestra.run.smithi116.stdout: debug mgr = 20 2020-11-20T13:55:32.531 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T13:55:32.531 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T13:55:32.531 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T13:55:32.531 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T13:55:32.532 INFO:teuthology.orchestra.run.smithi116.stdout: mgr/telemetry/nag = false 2020-11-20T13:55:32.532 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T13:55:32.532 INFO:teuthology.orchestra.run.smithi116.stdout:[mon] 2020-11-20T13:55:32.532 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T13:55:32.532 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T13:55:32.533 INFO:teuthology.orchestra.run.smithi116.stdout: debug paxos = 20 2020-11-20T13:55:32.533 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T13:55:32.533 INFO:teuthology.orchestra.run.smithi116.stdout: mon data avail warn = 5 2020-11-20T13:55:32.533 INFO:teuthology.orchestra.run.smithi116.stdout: mon mgr mkfs grace = 240 2020-11-20T13:55:32.534 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T13:55:32.534 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd reporter subtree level = osd 2020-11-20T13:55:32.534 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd prime pg temp = true 2020-11-20T13:55:32.534 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T13:55:32.534 INFO:teuthology.orchestra.run.smithi116.stdout: mon op complaint time = 120 2020-11-20T13:55:32.534 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T13:55:32.535 INFO:teuthology.orchestra.run.smithi116.stdout:[client] 2020-11-20T13:55:32.535 INFO:teuthology.orchestra.run.smithi116.stdout: rgw cache enabled = true 2020-11-20T13:55:32.535 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable ops log = true 2020-11-20T13:55:32.535 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable usage log = true 2020-11-20T13:55:32.535 INFO:teuthology.orchestra.run.smithi116.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T13:55:32.536 INFO:teuthology.orchestra.run.smithi116.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T13:55:32.536 INFO:teuthology.orchestra.run.smithi116.stdout: client mount timeout = 600 2020-11-20T13:55:32.536 INFO:teuthology.orchestra.run.smithi116.stdout: debug client = 20 2020-11-20T13:55:32.536 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T13:55:32.537 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = "" 2020-11-20T13:55:32.537 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T13:55:32.537 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T13:55:32.537 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.a] 2020-11-20T13:55:32.537 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.b] 2020-11-20T13:55:32.538 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.c] 2020-11-20T13:55:32.538 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.a] 2020-11-20T13:55:32.538 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.b] 2020-11-20T13:55:32.538 INFO:teuthology.orchestra.run.smithi116.stdout:[mds] 2020-11-20T13:55:32.538 INFO:teuthology.orchestra.run.smithi116.stdout: debug mds = 20 2020-11-20T13:55:32.539 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T13:55:32.539 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal frag = True 2020-11-20T13:55:32.539 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal fragment size max = 10000 2020-11-20T13:55:32.539 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal merge size = 5 2020-11-20T13:55:32.539 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split bits = 3 2020-11-20T13:55:32.540 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split size = 100 2020-11-20T13:55:32.540 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug frag = True 2020-11-20T13:55:32.540 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug scatterstat = True 2020-11-20T13:55:32.540 INFO:teuthology.orchestra.run.smithi116.stdout: mds op complaint time = 180 2020-11-20T13:55:32.540 INFO:teuthology.orchestra.run.smithi116.stdout: mds verify scatter = True 2020-11-20T13:55:32.541 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T13:55:32.541 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T13:55:32.541 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T13:55:32.543 INFO:teuthology.orchestra.run.smithi040.stdout:[global] 2020-11-20T13:55:32.543 INFO:teuthology.orchestra.run.smithi040.stdout: chdir = "" 2020-11-20T13:55:32.543 INFO:teuthology.orchestra.run.smithi040.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T13:55:32.543 INFO:teuthology.orchestra.run.smithi040.stdout: auth supported = cephx 2020-11-20T13:55:32.544 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:55:32.544 INFO:teuthology.orchestra.run.smithi040.stdout: filestore xattr use omap = true 2020-11-20T13:55:32.544 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:55:32.544 INFO:teuthology.orchestra.run.smithi040.stdout: mon clock drift allowed = 1.000 2020-11-20T13:55:32.544 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:55:32.545 INFO:teuthology.orchestra.run.smithi040.stdout: osd crush chooseleaf type = 0 2020-11-20T13:55:32.545 INFO:teuthology.orchestra.run.smithi040.stdout: auth debug = true 2020-11-20T13:55:32.545 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:55:32.545 INFO:teuthology.orchestra.run.smithi040.stdout: ms die on old message = true 2020-11-20T13:55:32.545 INFO:teuthology.orchestra.run.smithi040.stdout: ms die on bug = true 2020-11-20T13:55:32.546 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:55:32.546 INFO:teuthology.orchestra.run.smithi040.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T13:55:32.546 INFO:teuthology.orchestra.run.smithi040.stdout: mon pg warn max object skew = 0 2020-11-20T13:55:32.546 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:55:32.546 INFO:teuthology.orchestra.run.smithi040.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T13:55:32.547 INFO:teuthology.orchestra.run.smithi040.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T13:55:32.547 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:55:32.547 INFO:teuthology.orchestra.run.smithi040.stdout: osd pool default size = 2 2020-11-20T13:55:32.547 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:55:32.547 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd allow primary affinity = true 2020-11-20T13:55:32.548 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd allow pg remap = true 2020-11-20T13:55:32.548 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on legacy crush tunables = false 2020-11-20T13:55:32.548 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on crush straw calc version zero = false 2020-11-20T13:55:32.548 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on no sortbitwise = false 2020-11-20T13:55:32.548 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on osd down out interval zero = false 2020-11-20T13:55:32.549 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on too few osds = false 2020-11-20T13:55:32.549 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T13:55:32.549 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T13:55:32.549 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:55:32.549 INFO:teuthology.orchestra.run.smithi040.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T13:55:32.550 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:55:32.550 INFO:teuthology.orchestra.run.smithi040.stdout: osd default data pool replay window = 5 2020-11-20T13:55:32.550 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:55:32.550 INFO:teuthology.orchestra.run.smithi040.stdout: mon allow pool delete = true 2020-11-20T13:55:32.550 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:55:32.551 INFO:teuthology.orchestra.run.smithi040.stdout: mon cluster log file level = debug 2020-11-20T13:55:32.551 INFO:teuthology.orchestra.run.smithi040.stdout: debug asserts on shutdown = true 2020-11-20T13:55:32.551 INFO:teuthology.orchestra.run.smithi040.stdout: mon host = "172.21.15.40,172.21.15.116,[v2:172.21.15.116:3301,v1:172.21.15.116:6790]" 2020-11-20T13:55:32.551 INFO:teuthology.orchestra.run.smithi040.stdout: fsid = da1ed737-5174-430f-a3c9-56f6bbfad13c 2020-11-20T13:55:32.551 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:55:32.552 INFO:teuthology.orchestra.run.smithi040.stdout:[osd] 2020-11-20T13:55:32.552 INFO:teuthology.orchestra.run.smithi040.stdout: osd journal size = 100 2020-11-20T13:55:32.552 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:55:32.552 INFO:teuthology.orchestra.run.smithi040.stdout: osd scrub load threshold = 5.0 2020-11-20T13:55:32.552 INFO:teuthology.orchestra.run.smithi040.stdout: osd scrub max interval = 600 2020-11-20T13:55:32.553 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:55:32.553 INFO:teuthology.orchestra.run.smithi040.stdout: osd recover clone overlap = true 2020-11-20T13:55:32.553 INFO:teuthology.orchestra.run.smithi040.stdout: osd recovery max chunk = 1048576 2020-11-20T13:55:32.553 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:55:32.554 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug shutdown = true 2020-11-20T13:55:32.554 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug op order = true 2020-11-20T13:55:32.554 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug verify stray on activate = true 2020-11-20T13:55:32.554 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:55:32.554 INFO:teuthology.orchestra.run.smithi040.stdout: osd open classes on start = true 2020-11-20T13:55:32.555 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug pg log writeout = true 2020-11-20T13:55:32.555 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:55:32.555 INFO:teuthology.orchestra.run.smithi040.stdout: osd deep scrub update digest min age = 30 2020-11-20T13:55:32.555 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:55:32.555 INFO:teuthology.orchestra.run.smithi040.stdout: osd map max advance = 10 2020-11-20T13:55:32.556 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:55:32.556 INFO:teuthology.orchestra.run.smithi040.stdout: journal zero on create = true 2020-11-20T13:55:32.556 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:55:32.556 INFO:teuthology.orchestra.run.smithi040.stdout: filestore ondisk finisher threads = 3 2020-11-20T13:55:32.556 INFO:teuthology.orchestra.run.smithi040.stdout: filestore apply finisher threads = 3 2020-11-20T13:55:32.557 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:55:32.557 INFO:teuthology.orchestra.run.smithi040.stdout: bdev debug aio = true 2020-11-20T13:55:32.557 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug misdirected ops = true 2020-11-20T13:55:32.557 INFO:teuthology.orchestra.run.smithi040.stdout: bluestore block size = 96636764160 2020-11-20T13:55:32.558 INFO:teuthology.orchestra.run.smithi040.stdout: bluestore fsck on mount = True 2020-11-20T13:55:32.558 INFO:teuthology.orchestra.run.smithi040.stdout: debug bluefs = 20 2020-11-20T13:55:32.558 INFO:teuthology.orchestra.run.smithi040.stdout: debug bluestore = 20 2020-11-20T13:55:32.558 INFO:teuthology.orchestra.run.smithi040.stdout: debug filestore = 20 2020-11-20T13:55:32.558 INFO:teuthology.orchestra.run.smithi040.stdout: debug journal = 20 2020-11-20T13:55:32.559 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 20 2020-11-20T13:55:32.559 INFO:teuthology.orchestra.run.smithi040.stdout: debug osd = 25 2020-11-20T13:55:32.559 INFO:teuthology.orchestra.run.smithi040.stdout: debug rocksdb = 10 2020-11-20T13:55:32.559 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T13:55:32.559 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd full ratio = 0.9 2020-11-20T13:55:32.560 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd nearfull ratio = 0.8 2020-11-20T13:55:32.560 INFO:teuthology.orchestra.run.smithi040.stdout: osd failsafe full ratio = 0.95 2020-11-20T13:55:32.560 INFO:teuthology.orchestra.run.smithi040.stdout: osd objectstore = bluestore 2020-11-20T13:55:32.560 INFO:teuthology.orchestra.run.smithi040.stdout: osd op complaint time = 180 2020-11-20T13:55:32.560 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:55:32.561 INFO:teuthology.orchestra.run.smithi040.stdout:[mgr] 2020-11-20T13:55:32.561 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2020-11-20T13:55:32.561 INFO:teuthology.orchestra.run.smithi040.stdout: debug mgr = 20 2020-11-20T13:55:32.561 INFO:teuthology.orchestra.run.smithi040.stdout: debug mon = 20 2020-11-20T13:55:32.561 INFO:teuthology.orchestra.run.smithi040.stdout: debug auth = 20 2020-11-20T13:55:32.562 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min pgs per osd = 4 2020-11-20T13:55:32.562 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min bytes per osd = 10 2020-11-20T13:55:32.562 INFO:teuthology.orchestra.run.smithi040.stdout: mgr/telemetry/nag = false 2020-11-20T13:55:32.562 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:55:32.562 INFO:teuthology.orchestra.run.smithi040.stdout:[mon] 2020-11-20T13:55:32.563 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2020-11-20T13:55:32.563 INFO:teuthology.orchestra.run.smithi040.stdout: debug mon = 20 2020-11-20T13:55:32.563 INFO:teuthology.orchestra.run.smithi040.stdout: debug paxos = 20 2020-11-20T13:55:32.563 INFO:teuthology.orchestra.run.smithi040.stdout: debug auth = 20 2020-11-20T13:55:32.564 INFO:teuthology.orchestra.run.smithi040.stdout: mon data avail warn = 5 2020-11-20T13:55:32.564 INFO:teuthology.orchestra.run.smithi040.stdout: mon mgr mkfs grace = 240 2020-11-20T13:55:32.564 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min pgs per osd = 4 2020-11-20T13:55:32.564 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd reporter subtree level = osd 2020-11-20T13:55:32.564 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd prime pg temp = true 2020-11-20T13:55:32.565 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min bytes per osd = 10 2020-11-20T13:55:32.565 INFO:teuthology.orchestra.run.smithi040.stdout: mon op complaint time = 120 2020-11-20T13:55:32.565 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:55:32.565 INFO:teuthology.orchestra.run.smithi040.stdout:[client] 2020-11-20T13:55:32.565 INFO:teuthology.orchestra.run.smithi040.stdout: rgw cache enabled = true 2020-11-20T13:55:32.566 INFO:teuthology.orchestra.run.smithi040.stdout: rgw enable ops log = true 2020-11-20T13:55:32.566 INFO:teuthology.orchestra.run.smithi040.stdout: rgw enable usage log = true 2020-11-20T13:55:32.566 INFO:teuthology.orchestra.run.smithi040.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T13:55:32.566 INFO:teuthology.orchestra.run.smithi040.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T13:55:32.566 INFO:teuthology.orchestra.run.smithi040.stdout: client mount timeout = 600 2020-11-20T13:55:32.567 INFO:teuthology.orchestra.run.smithi040.stdout: debug client = 20 2020-11-20T13:55:32.567 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2020-11-20T13:55:32.567 INFO:teuthology.orchestra.run.smithi040.stdout: pid file = "" 2020-11-20T13:55:32.567 INFO:teuthology.orchestra.run.smithi040.stdout: rados mon op timeout = 15m 2020-11-20T13:55:32.567 INFO:teuthology.orchestra.run.smithi040.stdout: rados osd op timeout = 15m 2020-11-20T13:55:32.567 INFO:teuthology.orchestra.run.smithi040.stdout:[mon.a] 2020-11-20T13:55:32.568 INFO:teuthology.orchestra.run.smithi040.stdout:[mon.b] 2020-11-20T13:55:32.568 INFO:teuthology.orchestra.run.smithi040.stdout:[mon.c] 2020-11-20T13:55:32.568 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.a] 2020-11-20T13:55:32.568 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.b] 2020-11-20T13:55:32.568 INFO:teuthology.orchestra.run.smithi040.stdout:[mds] 2020-11-20T13:55:32.569 INFO:teuthology.orchestra.run.smithi040.stdout: debug mds = 20 2020-11-20T13:55:32.569 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2020-11-20T13:55:32.569 INFO:teuthology.orchestra.run.smithi040.stdout: mds bal frag = True 2020-11-20T13:55:32.569 INFO:teuthology.orchestra.run.smithi040.stdout: mds bal fragment size max = 10000 2020-11-20T13:55:32.569 INFO:teuthology.orchestra.run.smithi040.stdout: mds bal merge size = 5 2020-11-20T13:55:32.570 INFO:teuthology.orchestra.run.smithi040.stdout: mds bal split bits = 3 2020-11-20T13:55:32.570 INFO:teuthology.orchestra.run.smithi040.stdout: mds bal split size = 100 2020-11-20T13:55:32.570 INFO:teuthology.orchestra.run.smithi040.stdout: mds debug frag = True 2020-11-20T13:55:32.570 INFO:teuthology.orchestra.run.smithi040.stdout: mds debug scatterstat = True 2020-11-20T13:55:32.570 INFO:teuthology.orchestra.run.smithi040.stdout: mds op complaint time = 180 2020-11-20T13:55:32.571 INFO:teuthology.orchestra.run.smithi040.stdout: mds verify scatter = True 2020-11-20T13:55:32.571 INFO:teuthology.orchestra.run.smithi040.stdout: osd op complaint time = 180 2020-11-20T13:55:32.571 INFO:teuthology.orchestra.run.smithi040.stdout: rados mon op timeout = 15m 2020-11-20T13:55:32.571 INFO:teuthology.orchestra.run.smithi040.stdout: rados osd op timeout = 15m 2020-11-20T13:55:32.572 INFO:tasks.ceph:Creating admin key on mon.a... 2020-11-20T13:55:32.572 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2020-11-20T13:55:32.628 INFO:tasks.ceph:Copying monmap to all nodes... 2020-11-20T13:55:32.645 DEBUG:teuthology.orchestra.remote:smithi040:/etc/ceph/ceph.keyring is 206B 2020-11-20T13:55:32.742 DEBUG:teuthology.orchestra.remote:smithi040:/home/ubuntu/cephtest/ceph.monmap is 379B 2020-11-20T13:55:32.790 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi040.front.sepia.ceph.com 2020-11-20T13:55:32.791 INFO:teuthology.orchestra.run.smithi040:> set -ex 2020-11-20T13:55:32.791 INFO:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/ceph/ceph.keyring 2020-11-20T13:55:32.791 INFO:teuthology.orchestra.run.smithi040:> sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-20T13:55:32.804 INFO:teuthology.orchestra.run.smithi040.stderr:+ sudo dd of=/etc/ceph/ceph.keyring 2020-11-20T13:55:32.821 INFO:teuthology.orchestra.run.smithi040.stderr:0+1 records in 2020-11-20T13:55:32.821 INFO:teuthology.orchestra.run.smithi040.stderr:0+1 records out 2020-11-20T13:55:32.822 INFO:teuthology.orchestra.run.smithi040.stderr:206 bytes copied, 4.2659e-05 s, 4.8 MB/s 2020-11-20T13:55:32.822 INFO:teuthology.orchestra.run.smithi040.stderr:+ sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-20T13:55:32.839 INFO:teuthology.orchestra.run.smithi040:> set -ex 2020-11-20T13:55:32.839 INFO:teuthology.orchestra.run.smithi040:> dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-20T13:55:32.892 INFO:teuthology.orchestra.run.smithi040.stderr:+ dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-20T13:55:32.893 INFO:teuthology.orchestra.run.smithi040.stderr:0+1 records in 2020-11-20T13:55:32.893 INFO:teuthology.orchestra.run.smithi040.stderr:0+1 records out 2020-11-20T13:55:32.894 INFO:teuthology.orchestra.run.smithi040.stderr:379 bytes copied, 5.1696e-05 s, 7.3 MB/s 2020-11-20T13:55:32.894 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi116.front.sepia.ceph.com 2020-11-20T13:55:32.894 INFO:teuthology.orchestra.run.smithi116:> set -ex 2020-11-20T13:55:32.895 INFO:teuthology.orchestra.run.smithi116:> sudo dd of=/etc/ceph/ceph.keyring 2020-11-20T13:55:32.895 INFO:teuthology.orchestra.run.smithi116:> sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-20T13:55:32.912 INFO:teuthology.orchestra.run.smithi116.stderr:+ sudo dd of=/etc/ceph/ceph.keyring 2020-11-20T13:55:32.926 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records in 2020-11-20T13:55:32.926 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records out 2020-11-20T13:55:32.926 INFO:teuthology.orchestra.run.smithi116.stderr:206 bytes copied, 3.1903e-05 s, 6.5 MB/s 2020-11-20T13:55:32.927 INFO:teuthology.orchestra.run.smithi116.stderr:+ sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-20T13:55:32.941 INFO:teuthology.orchestra.run.smithi116:> set -ex 2020-11-20T13:55:32.942 INFO:teuthology.orchestra.run.smithi116:> dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-20T13:55:32.995 INFO:teuthology.orchestra.run.smithi116.stderr:+ dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-20T13:55:32.996 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records in 2020-11-20T13:55:32.997 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records out 2020-11-20T13:55:32.997 INFO:teuthology.orchestra.run.smithi116.stderr:379 bytes copied, 3.1093e-05 s, 12.2 MB/s 2020-11-20T13:55:32.997 INFO:tasks.ceph:Setting up mon nodes... 2020-11-20T13:55:32.998 INFO:tasks.ceph:Setting up mgr nodes... 2020-11-20T13:55:32.998 INFO:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2020-11-20T13:55:33.058 INFO:teuthology.orchestra.run.smithi040.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2020-11-20T13:55:33.061 INFO:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2020-11-20T13:55:33.125 INFO:teuthology.orchestra.run.smithi116.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2020-11-20T13:55:33.127 INFO:tasks.ceph:Setting up mds nodes... 2020-11-20T13:55:33.128 INFO:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2020-11-20T13:55:33.189 INFO:teuthology.orchestra.run.smithi040.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2020-11-20T13:55:33.191 INFO:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2020-11-20T13:55:33.222 INFO:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2020-11-20T13:55:33.277 INFO:teuthology.orchestra.run.smithi116.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2020-11-20T13:55:33.279 INFO:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2020-11-20T13:55:33.306 INFO:tasks.ceph_client:Setting up client nodes... 2020-11-20T13:55:33.307 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2020-11-20T13:55:33.350 INFO:teuthology.orchestra.run.smithi040.stdout:creating /etc/ceph/ceph.client.0.keyring 2020-11-20T13:55:33.370 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2020-11-20T13:55:33.413 INFO:teuthology.orchestra.run.smithi040.stdout:creating /etc/ceph/ceph.client.1.keyring 2020-11-20T13:55:33.432 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2020-11-20T13:55:33.471 INFO:teuthology.orchestra.run.smithi116.stdout:creating /etc/ceph/ceph.client.2.keyring 2020-11-20T13:55:33.487 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2020-11-20T13:55:33.566 INFO:teuthology.orchestra.run.smithi116.stdout:creating /etc/ceph/ceph.client.3.keyring 2020-11-20T13:55:33.582 INFO:tasks.ceph:Running mkfs on osd nodes... 2020-11-20T13:55:33.582 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi040.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi116.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2020-11-20T13:55:33.583 INFO:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2020-11-20T13:55:33.614 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-11-20T13:55:33.615 INFO:tasks.ceph:role: osd.0 2020-11-20T13:55:33.615 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi040.front.sepia.ceph.com 2020-11-20T13:55:33.615 INFO:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2020-11-20T13:55:34.147 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2020-11-20T13:55:34.147 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-20T13:55:34.147 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-11-20T13:55:34.148 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 2020-11-20T13:55:34.148 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-20T13:55:34.148 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2020-11-20T13:55:34.148 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-11-20T13:55:34.149 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-20T13:55:34.149 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-20T13:55:34.149 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-20T13:55:34.150 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2020-11-20T13:55:34.150 INFO:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2020-11-20T13:55:34.274 INFO:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2020-11-20T13:55:34.311 INFO:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2020-11-20T13:55:34.382 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-11-20T13:55:34.382 INFO:tasks.ceph:role: osd.1 2020-11-20T13:55:34.382 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi040.front.sepia.ceph.com 2020-11-20T13:55:34.383 INFO:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2020-11-20T13:55:34.888 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2020-11-20T13:55:34.888 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-20T13:55:34.889 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-11-20T13:55:34.889 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 2020-11-20T13:55:34.889 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-20T13:55:34.889 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2020-11-20T13:55:34.890 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-11-20T13:55:34.890 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-20T13:55:34.890 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-20T13:55:34.890 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-20T13:55:34.891 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2020-11-20T13:55:34.892 INFO:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2020-11-20T13:55:34.940 INFO:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2020-11-20T13:55:34.976 INFO:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2020-11-20T13:55:35.047 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-11-20T13:55:35.048 INFO:tasks.ceph:role: osd.2 2020-11-20T13:55:35.048 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi040.front.sepia.ceph.com 2020-11-20T13:55:35.049 INFO:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2020-11-20T13:55:35.591 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2020-11-20T13:55:35.591 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-20T13:55:35.591 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-11-20T13:55:35.592 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 2020-11-20T13:55:35.592 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-20T13:55:35.592 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2020-11-20T13:55:35.592 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-11-20T13:55:35.593 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-20T13:55:35.593 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-20T13:55:35.593 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-20T13:55:35.594 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2020-11-20T13:55:35.594 INFO:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2020-11-20T13:55:35.672 INFO:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2020-11-20T13:55:35.708 INFO:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2020-11-20T13:55:35.782 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-11-20T13:55:35.782 INFO:tasks.ceph:role: osd.3 2020-11-20T13:55:35.783 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi040.front.sepia.ceph.com 2020-11-20T13:55:35.783 INFO:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2020-11-20T13:55:36.330 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2020-11-20T13:55:36.330 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-20T13:55:36.331 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-11-20T13:55:36.331 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 2020-11-20T13:55:36.331 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-20T13:55:36.331 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2020-11-20T13:55:36.332 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-11-20T13:55:36.332 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-20T13:55:36.332 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-20T13:55:36.332 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-20T13:55:36.333 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2020-11-20T13:55:36.334 INFO:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2020-11-20T13:55:36.447 INFO:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2020-11-20T13:55:36.484 INFO:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-20T13:55:36.576 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:55:36.575+0000 7f70cd8caec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2020-11-20T13:55:36.578 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:55:36.577+0000 7f70cd8caec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2020-11-20T13:55:36.578 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:55:36.577+0000 7f70cd8caec0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-11-20T13:55:36.579 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:55:36.577+0000 7f70cd8caec0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-11-20T13:55:36.579 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:55:36.577+0000 7f70cd8caec0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-11-20T13:55:36.585 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:55:36.584+0000 7f70cd8caec0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2020-11-20T13:55:36.608 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:55:36.604+0000 7f70cd8caec0 -1 freelist read_size_meta_from_db missing size meta in DB 2020-11-20T13:55:38.930 INFO:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2020-11-20T13:55:38.966 INFO:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-20T13:55:39.059 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:55:39.057+0000 7f0edc6f8ec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2020-11-20T13:55:39.059 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:55:39.058+0000 7f0edc6f8ec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2020-11-20T13:55:39.059 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:55:39.058+0000 7f0edc6f8ec0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-11-20T13:55:39.059 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:55:39.058+0000 7f0edc6f8ec0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-11-20T13:55:39.060 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:55:39.058+0000 7f0edc6f8ec0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-11-20T13:55:39.060 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:55:39.058+0000 7f0edc6f8ec0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2020-11-20T13:55:39.071 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:55:39.070+0000 7f0edc6f8ec0 -1 freelist read_size_meta_from_db missing size meta in DB 2020-11-20T13:55:41.404 INFO:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2020-11-20T13:55:41.440 INFO:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-20T13:55:41.536 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:55:41.534+0000 7f07d2c26ec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2020-11-20T13:55:41.536 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:55:41.535+0000 7f07d2c26ec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2020-11-20T13:55:41.536 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:55:41.535+0000 7f07d2c26ec0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-11-20T13:55:41.537 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:55:41.535+0000 7f07d2c26ec0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-11-20T13:55:41.537 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:55:41.535+0000 7f07d2c26ec0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-11-20T13:55:41.537 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:55:41.535+0000 7f07d2c26ec0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2020-11-20T13:55:41.550 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:55:41.549+0000 7f07d2c26ec0 -1 freelist read_size_meta_from_db missing size meta in DB 2020-11-20T13:55:43.875 INFO:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2020-11-20T13:55:43.912 INFO:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-20T13:55:44.007 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:55:44.006+0000 7f3a73cc1ec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2020-11-20T13:55:44.008 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:55:44.006+0000 7f3a73cc1ec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2020-11-20T13:55:44.008 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:55:44.006+0000 7f3a73cc1ec0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2020-11-20T13:55:44.008 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:55:44.006+0000 7f3a73cc1ec0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2020-11-20T13:55:44.008 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:55:44.006+0000 7f3a73cc1ec0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2020-11-20T13:55:44.009 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:55:44.007+0000 7f3a73cc1ec0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2020-11-20T13:55:44.021 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:55:44.020+0000 7f3a73cc1ec0 -1 freelist read_size_meta_from_db missing size meta in DB 2020-11-20T13:55:46.352 INFO:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2020-11-20T13:55:46.390 INFO:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2020-11-20T13:55:46.423 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-11-20T13:55:46.423 INFO:tasks.ceph:role: osd.4 2020-11-20T13:55:46.423 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi116.front.sepia.ceph.com 2020-11-20T13:55:46.424 INFO:teuthology.orchestra.run.smithi116:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2020-11-20T13:55:46.975 INFO:teuthology.orchestra.run.smithi116.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2020-11-20T13:55:46.976 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-20T13:55:46.976 INFO:teuthology.orchestra.run.smithi116.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-11-20T13:55:46.976 INFO:teuthology.orchestra.run.smithi116.stdout: = reflink=1 2020-11-20T13:55:46.976 INFO:teuthology.orchestra.run.smithi116.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-20T13:55:46.977 INFO:teuthology.orchestra.run.smithi116.stdout: = sunit=0 swidth=0 blks 2020-11-20T13:55:46.977 INFO:teuthology.orchestra.run.smithi116.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-11-20T13:55:46.977 INFO:teuthology.orchestra.run.smithi116.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-20T13:55:46.977 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-20T13:55:46.978 INFO:teuthology.orchestra.run.smithi116.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-20T13:55:46.978 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi116.front.sepia.ceph.com -o noatime 2020-11-20T13:55:46.979 INFO:teuthology.orchestra.run.smithi116:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2020-11-20T13:55:47.099 INFO:teuthology.orchestra.run.smithi116:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2020-11-20T13:55:47.132 INFO:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2020-11-20T13:55:47.200 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-11-20T13:55:47.200 INFO:tasks.ceph:role: osd.5 2020-11-20T13:55:47.201 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi116.front.sepia.ceph.com 2020-11-20T13:55:47.201 INFO:teuthology.orchestra.run.smithi116:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2020-11-20T13:55:47.712 INFO:teuthology.orchestra.run.smithi116.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2020-11-20T13:55:47.712 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-20T13:55:47.712 INFO:teuthology.orchestra.run.smithi116.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-11-20T13:55:47.713 INFO:teuthology.orchestra.run.smithi116.stdout: = reflink=1 2020-11-20T13:55:47.713 INFO:teuthology.orchestra.run.smithi116.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-20T13:55:47.713 INFO:teuthology.orchestra.run.smithi116.stdout: = sunit=0 swidth=0 blks 2020-11-20T13:55:47.713 INFO:teuthology.orchestra.run.smithi116.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-11-20T13:55:47.714 INFO:teuthology.orchestra.run.smithi116.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-20T13:55:47.714 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-20T13:55:47.714 INFO:teuthology.orchestra.run.smithi116.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-20T13:55:47.715 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi116.front.sepia.ceph.com -o noatime 2020-11-20T13:55:47.716 INFO:teuthology.orchestra.run.smithi116:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2020-11-20T13:55:47.762 INFO:teuthology.orchestra.run.smithi116:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2020-11-20T13:55:47.793 INFO:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2020-11-20T13:55:47.861 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-11-20T13:55:47.862 INFO:tasks.ceph:role: osd.6 2020-11-20T13:55:47.862 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi116.front.sepia.ceph.com 2020-11-20T13:55:47.862 INFO:teuthology.orchestra.run.smithi116:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2020-11-20T13:55:48.403 INFO:teuthology.orchestra.run.smithi116.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2020-11-20T13:55:48.403 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-20T13:55:48.403 INFO:teuthology.orchestra.run.smithi116.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-11-20T13:55:48.404 INFO:teuthology.orchestra.run.smithi116.stdout: = reflink=1 2020-11-20T13:55:48.404 INFO:teuthology.orchestra.run.smithi116.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-20T13:55:48.404 INFO:teuthology.orchestra.run.smithi116.stdout: = sunit=0 swidth=0 blks 2020-11-20T13:55:48.404 INFO:teuthology.orchestra.run.smithi116.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-11-20T13:55:48.404 INFO:teuthology.orchestra.run.smithi116.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-20T13:55:48.405 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-20T13:55:48.405 INFO:teuthology.orchestra.run.smithi116.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-20T13:55:48.406 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi116.front.sepia.ceph.com -o noatime 2020-11-20T13:55:48.406 INFO:teuthology.orchestra.run.smithi116:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2020-11-20T13:55:48.475 INFO:teuthology.orchestra.run.smithi116:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2020-11-20T13:55:48.507 INFO:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2020-11-20T13:55:48.574 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-11-20T13:55:48.574 INFO:tasks.ceph:role: osd.7 2020-11-20T13:55:48.575 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi116.front.sepia.ceph.com 2020-11-20T13:55:48.575 INFO:teuthology.orchestra.run.smithi116:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2020-11-20T13:55:49.105 INFO:teuthology.orchestra.run.smithi116.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2020-11-20T13:55:49.106 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-20T13:55:49.106 INFO:teuthology.orchestra.run.smithi116.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-11-20T13:55:49.106 INFO:teuthology.orchestra.run.smithi116.stdout: = reflink=1 2020-11-20T13:55:49.107 INFO:teuthology.orchestra.run.smithi116.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-20T13:55:49.107 INFO:teuthology.orchestra.run.smithi116.stdout: = sunit=0 swidth=0 blks 2020-11-20T13:55:49.107 INFO:teuthology.orchestra.run.smithi116.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-11-20T13:55:49.107 INFO:teuthology.orchestra.run.smithi116.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-20T13:55:49.107 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-20T13:55:49.108 INFO:teuthology.orchestra.run.smithi116.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-20T13:55:49.109 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi116.front.sepia.ceph.com -o noatime 2020-11-20T13:55:49.109 INFO:teuthology.orchestra.run.smithi116:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2020-11-20T13:55:49.214 INFO:teuthology.orchestra.run.smithi116:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2020-11-20T13:55:49.246 INFO:teuthology.orchestra.run.smithi116:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-20T13:55:49.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:55:49.329+0000 7f943b29aec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2020-11-20T13:55:49.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:55:49.329+0000 7f943b29aec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2020-11-20T13:55:49.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:55:49.329+0000 7f943b29aec0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2020-11-20T13:55:49.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:55:49.329+0000 7f943b29aec0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2020-11-20T13:55:49.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:55:49.329+0000 7f943b29aec0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2020-11-20T13:55:49.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:55:49.335+0000 7f943b29aec0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2020-11-20T13:55:49.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:55:49.358+0000 7f943b29aec0 -1 freelist read_size_meta_from_db missing size meta in DB 2020-11-20T13:55:51.672 INFO:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2020-11-20T13:55:51.704 INFO:teuthology.orchestra.run.smithi116:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-20T13:55:51.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:55:51.792+0000 7fb5152d4ec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2020-11-20T13:55:51.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:55:51.792+0000 7fb5152d4ec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2020-11-20T13:55:51.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:55:51.792+0000 7fb5152d4ec0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2020-11-20T13:55:51.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:55:51.792+0000 7fb5152d4ec0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2020-11-20T13:55:51.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:55:51.792+0000 7fb5152d4ec0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2020-11-20T13:55:51.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:55:51.793+0000 7fb5152d4ec0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2020-11-20T13:55:51.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:55:51.804+0000 7fb5152d4ec0 -1 freelist read_size_meta_from_db missing size meta in DB 2020-11-20T13:55:54.138 INFO:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2020-11-20T13:55:54.169 INFO:teuthology.orchestra.run.smithi116:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-20T13:55:54.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:55:54.255+0000 7fbf40b16ec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2020-11-20T13:55:54.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:55:54.255+0000 7fbf40b16ec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2020-11-20T13:55:54.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:55:54.255+0000 7fbf40b16ec0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2020-11-20T13:55:54.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:55:54.255+0000 7fbf40b16ec0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2020-11-20T13:55:54.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:55:54.255+0000 7fbf40b16ec0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2020-11-20T13:55:54.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:55:54.256+0000 7fbf40b16ec0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2020-11-20T13:55:54.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:55:54.269+0000 7fbf40b16ec0 -1 freelist read_size_meta_from_db missing size meta in DB 2020-11-20T13:55:56.598 INFO:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2020-11-20T13:55:56.629 INFO:teuthology.orchestra.run.smithi116:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-20T13:55:56.717 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:55:56.716+0000 7f51eda7dec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2020-11-20T13:55:56.718 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:55:56.717+0000 7f51eda7dec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2020-11-20T13:55:56.718 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:55:56.717+0000 7f51eda7dec0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2020-11-20T13:55:56.718 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:55:56.717+0000 7f51eda7dec0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2020-11-20T13:55:56.718 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:55:56.717+0000 7f51eda7dec0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2020-11-20T13:55:56.719 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:55:56.717+0000 7f51eda7dec0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2020-11-20T13:55:56.731 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:55:56.731+0000 7f51eda7dec0 -1 freelist read_size_meta_from_db missing size meta in DB 2020-11-20T13:55:59.060 INFO:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2020-11-20T13:55:59.091 INFO:tasks.ceph:Reading keys from all nodes... 2020-11-20T13:55:59.092 INFO:teuthology.orchestra.run.smithi040:> mktemp 2020-11-20T13:55:59.110 INFO:teuthology.orchestra.run.smithi040.stdout:/tmp/tmp.FjVKZM9Bn4 2020-11-20T13:55:59.111 INFO:teuthology.orchestra.run.smithi040:> sudo cp /var/lib/ceph/mgr/ceph-y/keyring /tmp/tmp.FjVKZM9Bn4 2020-11-20T13:55:59.188 INFO:teuthology.orchestra.run.smithi040:> sudo chmod 0666 /tmp/tmp.FjVKZM9Bn4 2020-11-20T13:55:59.320 DEBUG:teuthology.orchestra.remote:smithi040:/tmp/tmp.FjVKZM9Bn4 is 56B 2020-11-20T13:55:59.341 INFO:teuthology.orchestra.run.smithi040:> rm -fr /tmp/tmp.FjVKZM9Bn4 2020-11-20T13:55:59.356 INFO:teuthology.orchestra.run.smithi040:> mktemp 2020-11-20T13:55:59.410 INFO:teuthology.orchestra.run.smithi040.stdout:/tmp/tmp.zjmkONrm0K 2020-11-20T13:55:59.411 INFO:teuthology.orchestra.run.smithi040:> sudo cp /var/lib/ceph/mds/ceph-a/keyring /tmp/tmp.zjmkONrm0K 2020-11-20T13:55:59.483 INFO:teuthology.orchestra.run.smithi040:> sudo chmod 0666 /tmp/tmp.zjmkONrm0K 2020-11-20T13:55:59.613 DEBUG:teuthology.orchestra.remote:smithi040:/tmp/tmp.zjmkONrm0K is 56B 2020-11-20T13:55:59.663 INFO:teuthology.orchestra.run.smithi040:> rm -fr /tmp/tmp.zjmkONrm0K 2020-11-20T13:55:59.679 INFO:teuthology.orchestra.run.smithi040:> mktemp 2020-11-20T13:55:59.735 INFO:teuthology.orchestra.run.smithi040.stdout:/tmp/tmp.wXPydcj42r 2020-11-20T13:55:59.736 INFO:teuthology.orchestra.run.smithi040:> sudo cp /var/lib/ceph/osd/ceph-0/keyring /tmp/tmp.wXPydcj42r 2020-11-20T13:55:59.809 INFO:teuthology.orchestra.run.smithi040:> sudo chmod 0666 /tmp/tmp.wXPydcj42r 2020-11-20T13:55:59.939 DEBUG:teuthology.orchestra.remote:smithi040:/tmp/tmp.wXPydcj42r is 56B 2020-11-20T13:55:59.987 INFO:teuthology.orchestra.run.smithi040:> rm -fr /tmp/tmp.wXPydcj42r 2020-11-20T13:56:00.002 INFO:teuthology.orchestra.run.smithi040:> mktemp 2020-11-20T13:56:00.056 INFO:teuthology.orchestra.run.smithi040.stdout:/tmp/tmp.JtaANqWGLN 2020-11-20T13:56:00.057 INFO:teuthology.orchestra.run.smithi040:> sudo cp /var/lib/ceph/osd/ceph-1/keyring /tmp/tmp.JtaANqWGLN 2020-11-20T13:56:00.129 INFO:teuthology.orchestra.run.smithi040:> sudo chmod 0666 /tmp/tmp.JtaANqWGLN 2020-11-20T13:56:00.274 DEBUG:teuthology.orchestra.remote:smithi040:/tmp/tmp.JtaANqWGLN is 56B 2020-11-20T13:56:00.325 INFO:teuthology.orchestra.run.smithi040:> rm -fr /tmp/tmp.JtaANqWGLN 2020-11-20T13:56:00.341 INFO:teuthology.orchestra.run.smithi040:> mktemp 2020-11-20T13:56:00.394 INFO:teuthology.orchestra.run.smithi040.stdout:/tmp/tmp.WuAk6bLQfg 2020-11-20T13:56:00.394 INFO:teuthology.orchestra.run.smithi040:> sudo cp /var/lib/ceph/osd/ceph-2/keyring /tmp/tmp.WuAk6bLQfg 2020-11-20T13:56:00.464 INFO:teuthology.orchestra.run.smithi040:> sudo chmod 0666 /tmp/tmp.WuAk6bLQfg 2020-11-20T13:56:00.592 DEBUG:teuthology.orchestra.remote:smithi040:/tmp/tmp.WuAk6bLQfg is 56B 2020-11-20T13:56:00.639 INFO:teuthology.orchestra.run.smithi040:> rm -fr /tmp/tmp.WuAk6bLQfg 2020-11-20T13:56:00.654 INFO:teuthology.orchestra.run.smithi040:> mktemp 2020-11-20T13:56:00.708 INFO:teuthology.orchestra.run.smithi040.stdout:/tmp/tmp.rRSV62JnXK 2020-11-20T13:56:00.709 INFO:teuthology.orchestra.run.smithi040:> sudo cp /var/lib/ceph/osd/ceph-3/keyring /tmp/tmp.rRSV62JnXK 2020-11-20T13:56:00.779 INFO:teuthology.orchestra.run.smithi040:> sudo chmod 0666 /tmp/tmp.rRSV62JnXK 2020-11-20T13:56:00.906 DEBUG:teuthology.orchestra.remote:smithi040:/tmp/tmp.rRSV62JnXK is 56B 2020-11-20T13:56:00.954 INFO:teuthology.orchestra.run.smithi040:> rm -fr /tmp/tmp.rRSV62JnXK 2020-11-20T13:56:00.969 INFO:teuthology.orchestra.run.smithi116:> mktemp 2020-11-20T13:56:00.985 INFO:teuthology.orchestra.run.smithi116.stdout:/tmp/tmp.S3XJIwSShy 2020-11-20T13:56:00.986 INFO:teuthology.orchestra.run.smithi116:> sudo cp /var/lib/ceph/mgr/ceph-x/keyring /tmp/tmp.S3XJIwSShy 2020-11-20T13:56:01.057 INFO:teuthology.orchestra.run.smithi116:> sudo chmod 0666 /tmp/tmp.S3XJIwSShy 2020-11-20T13:56:01.188 DEBUG:teuthology.orchestra.remote:smithi116:/tmp/tmp.S3XJIwSShy is 56B 2020-11-20T13:56:01.238 INFO:teuthology.orchestra.run.smithi116:> rm -fr /tmp/tmp.S3XJIwSShy 2020-11-20T13:56:01.252 INFO:teuthology.orchestra.run.smithi116:> mktemp 2020-11-20T13:56:01.307 INFO:teuthology.orchestra.run.smithi116.stdout:/tmp/tmp.oPkaJ4KELE 2020-11-20T13:56:01.307 INFO:teuthology.orchestra.run.smithi116:> sudo cp /var/lib/ceph/mds/ceph-b/keyring /tmp/tmp.oPkaJ4KELE 2020-11-20T13:56:01.375 INFO:teuthology.orchestra.run.smithi116:> sudo chmod 0666 /tmp/tmp.oPkaJ4KELE 2020-11-20T13:56:01.422 INFO:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T13:56:01.424 INFO:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T13:56:01.529 DEBUG:teuthology.orchestra.remote:smithi116:/tmp/tmp.oPkaJ4KELE is 56B 2020-11-20T13:56:01.606 INFO:teuthology.orchestra.run.smithi116:> rm -fr /tmp/tmp.oPkaJ4KELE 2020-11-20T13:56:01.620 INFO:teuthology.orchestra.run.smithi116:> mktemp 2020-11-20T13:56:01.673 INFO:teuthology.orchestra.run.smithi116.stdout:/tmp/tmp.yKidcc7HR1 2020-11-20T13:56:01.674 INFO:teuthology.orchestra.run.smithi116:> sudo cp /var/lib/ceph/osd/ceph-4/keyring /tmp/tmp.yKidcc7HR1 2020-11-20T13:56:01.740 INFO:teuthology.orchestra.run.smithi116:> sudo chmod 0666 /tmp/tmp.yKidcc7HR1 2020-11-20T13:56:01.863 DEBUG:teuthology.orchestra.remote:smithi116:/tmp/tmp.yKidcc7HR1 is 56B 2020-11-20T13:56:01.911 INFO:teuthology.orchestra.run.smithi116:> rm -fr /tmp/tmp.yKidcc7HR1 2020-11-20T13:56:01.924 INFO:teuthology.orchestra.run.smithi116:> mktemp 2020-11-20T13:56:01.978 INFO:teuthology.orchestra.run.smithi116.stdout:/tmp/tmp.W5ORY1Csvx 2020-11-20T13:56:01.979 INFO:teuthology.orchestra.run.smithi116:> sudo cp /var/lib/ceph/osd/ceph-5/keyring /tmp/tmp.W5ORY1Csvx 2020-11-20T13:56:02.045 INFO:teuthology.orchestra.run.smithi116:> sudo chmod 0666 /tmp/tmp.W5ORY1Csvx 2020-11-20T13:56:02.170 DEBUG:teuthology.orchestra.remote:smithi116:/tmp/tmp.W5ORY1Csvx is 56B 2020-11-20T13:56:02.220 INFO:teuthology.orchestra.run.smithi116:> rm -fr /tmp/tmp.W5ORY1Csvx 2020-11-20T13:56:02.234 INFO:teuthology.orchestra.run.smithi116:> mktemp 2020-11-20T13:56:02.287 INFO:teuthology.orchestra.run.smithi116.stdout:/tmp/tmp.Oc5vWTKqTS 2020-11-20T13:56:02.288 INFO:teuthology.orchestra.run.smithi116:> sudo cp /var/lib/ceph/osd/ceph-6/keyring /tmp/tmp.Oc5vWTKqTS 2020-11-20T13:56:02.354 INFO:teuthology.orchestra.run.smithi116:> sudo chmod 0666 /tmp/tmp.Oc5vWTKqTS 2020-11-20T13:56:02.478 DEBUG:teuthology.orchestra.remote:smithi116:/tmp/tmp.Oc5vWTKqTS is 56B 2020-11-20T13:56:02.529 INFO:teuthology.orchestra.run.smithi116:> rm -fr /tmp/tmp.Oc5vWTKqTS 2020-11-20T13:56:02.543 INFO:teuthology.orchestra.run.smithi116:> mktemp 2020-11-20T13:56:02.596 INFO:teuthology.orchestra.run.smithi116.stdout:/tmp/tmp.FaGy3MrybE 2020-11-20T13:56:02.597 INFO:teuthology.orchestra.run.smithi116:> sudo cp /var/lib/ceph/osd/ceph-7/keyring /tmp/tmp.FaGy3MrybE 2020-11-20T13:56:02.663 INFO:teuthology.orchestra.run.smithi116:> sudo chmod 0666 /tmp/tmp.FaGy3MrybE 2020-11-20T13:56:02.787 DEBUG:teuthology.orchestra.remote:smithi116:/tmp/tmp.FaGy3MrybE is 56B 2020-11-20T13:56:02.835 INFO:teuthology.orchestra.run.smithi116:> rm -fr /tmp/tmp.FaGy3MrybE 2020-11-20T13:56:02.870 DEBUG:teuthology.orchestra.remote:smithi040:/etc/ceph/ceph.client.0.keyring is 59B 2020-11-20T13:56:02.965 DEBUG:teuthology.orchestra.remote:smithi040:/etc/ceph/ceph.client.1.keyring is 59B 2020-11-20T13:56:03.004 DEBUG:teuthology.orchestra.remote:smithi116:/etc/ceph/ceph.client.2.keyring is 59B 2020-11-20T13:56:03.099 DEBUG:teuthology.orchestra.remote:smithi116:/etc/ceph/ceph.client.3.keyring is 59B 2020-11-20T13:56:03.148 INFO:tasks.ceph:Adding keys to all mons... 2020-11-20T13:56:03.149 INFO:teuthology.orchestra.run.smithi040:> sudo tee -a /etc/ceph/ceph.keyring 2020-11-20T13:56:03.152 INFO:teuthology.orchestra.run.smithi116:> sudo tee -a /etc/ceph/ceph.keyring 2020-11-20T13:56:03.193 INFO:teuthology.orchestra.run.smithi040.stdout:[mgr.y] 2020-11-20T13:56:03.193 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQDVyrdfBgGJAxAAHLtRrJGJ4/Hhm8dh6OKvOw== 2020-11-20T13:56:03.194 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.a] 2020-11-20T13:56:03.194 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQDVyrdfonFLCxAArF1OF7FJCpDeppKb04UWAw== 2020-11-20T13:56:03.194 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.0] 2020-11-20T13:56:03.194 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQDYyrdfgXtcIhAABg9OfOB5wNYOfUuP+bQJSw== 2020-11-20T13:56:03.194 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.1] 2020-11-20T13:56:03.195 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQDbyrdfWk6FAxAAYo+smkJB36jCJNamuCOtFQ== 2020-11-20T13:56:03.195 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.2] 2020-11-20T13:56:03.195 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQDdyrdfN231HxAAz/2nTCIdltDe48S/sM/WIw== 2020-11-20T13:56:03.195 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.3] 2020-11-20T13:56:03.195 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQDgyrdfhTB3ABAAX1FyF2y3XlsWS5AiGKe0Cg== 2020-11-20T13:56:03.196 INFO:teuthology.orchestra.run.smithi040.stdout:[mgr.x] 2020-11-20T13:56:03.196 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQDVyrdfkHWGBxAAnIOJ1zPJ8Q0QLgXITiN3iQ== 2020-11-20T13:56:03.196 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.b] 2020-11-20T13:56:03.196 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQDVyrdfAieJEBAAbZUwBugtHZWiMGfvr3I1RQ== 2020-11-20T13:56:03.197 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.4] 2020-11-20T13:56:03.197 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQDlyrdf4xSqExAA+ZRs2OjQq5gbZn+kYRnvdQ== 2020-11-20T13:56:03.197 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.5] 2020-11-20T13:56:03.197 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQDnyrdfh7tKLxAAGqqcuya3izlwH6pA1Dywmw== 2020-11-20T13:56:03.197 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.6] 2020-11-20T13:56:03.198 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQDqyrdfhM5FDxAAR4pSp1zjt+xNgLCDt0G/Kw== 2020-11-20T13:56:03.198 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.7] 2020-11-20T13:56:03.198 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQDsyrdf4nHIKhAA5sKCctRhqCJ+uv4cQCr4lw== 2020-11-20T13:56:03.198 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2020-11-20T13:56:03.198 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQDVyrdfLzfxFBAAuJx4JTYvaoZNbURiP3iurA== 2020-11-20T13:56:03.199 INFO:teuthology.orchestra.run.smithi040.stdout:[client.1] 2020-11-20T13:56:03.199 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQDVyrdfdequGBAALzZYyNDFcNUoTpMqGd++5Q== 2020-11-20T13:56:03.199 INFO:teuthology.orchestra.run.smithi040.stdout:[client.2] 2020-11-20T13:56:03.199 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQDVyrdfsRgnHBAAjwSLX14/i3YXxUDgEcvyxg== 2020-11-20T13:56:03.200 INFO:teuthology.orchestra.run.smithi040.stdout:[client.3] 2020-11-20T13:56:03.200 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQDVyrdftp3QIRAAsODgAG8e4hSDiL3u3IAZtA== 2020-11-20T13:56:03.201 INFO:teuthology.orchestra.run.smithi116.stdout:[mgr.y] 2020-11-20T13:56:03.201 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDVyrdfBgGJAxAAHLtRrJGJ4/Hhm8dh6OKvOw== 2020-11-20T13:56:03.202 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.a] 2020-11-20T13:56:03.202 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDVyrdfonFLCxAArF1OF7FJCpDeppKb04UWAw== 2020-11-20T13:56:03.202 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.0] 2020-11-20T13:56:03.202 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDYyrdfgXtcIhAABg9OfOB5wNYOfUuP+bQJSw== 2020-11-20T13:56:03.202 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.1] 2020-11-20T13:56:03.203 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDbyrdfWk6FAxAAYo+smkJB36jCJNamuCOtFQ== 2020-11-20T13:56:03.203 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.2] 2020-11-20T13:56:03.203 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDdyrdfN231HxAAz/2nTCIdltDe48S/sM/WIw== 2020-11-20T13:56:03.203 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.3] 2020-11-20T13:56:03.203 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDgyrdfhTB3ABAAX1FyF2y3XlsWS5AiGKe0Cg== 2020-11-20T13:56:03.204 INFO:teuthology.orchestra.run.smithi116.stdout:[mgr.x] 2020-11-20T13:56:03.204 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDVyrdfkHWGBxAAnIOJ1zPJ8Q0QLgXITiN3iQ== 2020-11-20T13:56:03.204 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.b] 2020-11-20T13:56:03.204 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDVyrdfAieJEBAAbZUwBugtHZWiMGfvr3I1RQ== 2020-11-20T13:56:03.204 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.4] 2020-11-20T13:56:03.205 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDlyrdf4xSqExAA+ZRs2OjQq5gbZn+kYRnvdQ== 2020-11-20T13:56:03.205 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.5] 2020-11-20T13:56:03.205 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDnyrdfh7tKLxAAGqqcuya3izlwH6pA1Dywmw== 2020-11-20T13:56:03.205 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.6] 2020-11-20T13:56:03.205 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDqyrdfhM5FDxAAR4pSp1zjt+xNgLCDt0G/Kw== 2020-11-20T13:56:03.206 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.7] 2020-11-20T13:56:03.206 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDsyrdf4nHIKhAA5sKCctRhqCJ+uv4cQCr4lw== 2020-11-20T13:56:03.206 INFO:teuthology.orchestra.run.smithi116.stdout:[client.0] 2020-11-20T13:56:03.206 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDVyrdfLzfxFBAAuJx4JTYvaoZNbURiP3iurA== 2020-11-20T13:56:03.206 INFO:teuthology.orchestra.run.smithi116.stdout:[client.1] 2020-11-20T13:56:03.207 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDVyrdfdequGBAALzZYyNDFcNUoTpMqGd++5Q== 2020-11-20T13:56:03.207 INFO:teuthology.orchestra.run.smithi116.stdout:[client.2] 2020-11-20T13:56:03.207 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDVyrdfsRgnHBAAjwSLX14/i3YXxUDgEcvyxg== 2020-11-20T13:56:03.207 INFO:teuthology.orchestra.run.smithi116.stdout:[client.3] 2020-11-20T13:56:03.207 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDVyrdftp3QIRAAsODgAG8e4hSDiL3u3IAZtA== 2020-11-20T13:56:03.208 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-11-20T13:56:03.245 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-11-20T13:56:03.302 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-20T13:56:03.305 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-20T13:56:03.360 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T13:56:03.363 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T13:56:03.414 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T13:56:03.417 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T13:56:03.467 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T13:56:03.470 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T13:56:03.521 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T13:56:03.524 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T13:56:03.573 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-11-20T13:56:03.576 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-11-20T13:56:03.625 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-20T13:56:03.628 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-20T13:56:03.680 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T13:56:03.684 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T13:56:03.732 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T13:56:03.735 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T13:56:03.785 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T13:56:03.787 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T13:56:03.837 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T13:56:03.839 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T13:56:03.889 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-11-20T13:56:03.892 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-11-20T13:56:03.944 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-11-20T13:56:03.947 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-11-20T13:56:03.999 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-11-20T13:56:04.001 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-11-20T13:56:04.057 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-11-20T13:56:04.061 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-11-20T13:56:04.113 INFO:tasks.ceph:Running mkfs on mon nodes... 2020-11-20T13:56:04.113 INFO:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2020-11-20T13:56:04.144 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2020-11-20T13:56:04.242 INFO:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2020-11-20T13:56:04.274 INFO:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2020-11-20T13:56:04.301 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2020-11-20T13:56:04.395 INFO:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2020-11-20T13:56:04.422 INFO:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2020-11-20T13:56:04.493 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2020-11-20T13:56:04.585 INFO:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2020-11-20T13:56:04.612 INFO:teuthology.orchestra.run.smithi040:> rm -- /home/ubuntu/cephtest/ceph.monmap 2020-11-20T13:56:04.615 INFO:teuthology.orchestra.run.smithi116:> rm -- /home/ubuntu/cephtest/ceph.monmap 2020-11-20T13:56:04.666 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2020-11-20T13:56:04.667 INFO:tasks.ceph.mon.a:Restarting daemon 2020-11-20T13:56:04.667 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2020-11-20T13:56:04.670 INFO:tasks.ceph.mon.a:Started 2020-11-20T13:56:04.670 INFO:tasks.ceph.mon.b:Restarting daemon 2020-11-20T13:56:04.670 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2020-11-20T13:56:04.709 INFO:tasks.ceph.mon.b:Started 2020-11-20T13:56:04.709 INFO:tasks.ceph.mon.c:Restarting daemon 2020-11-20T13:56:04.710 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2020-11-20T13:56:04.718 INFO:tasks.ceph.mon.c:Started 2020-11-20T13:56:04.719 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2020-11-20T13:56:04.719 INFO:tasks.ceph.mgr.y:Restarting daemon 2020-11-20T13:56:04.720 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2020-11-20T13:56:04.722 INFO:tasks.ceph.mgr.y:Started 2020-11-20T13:56:04.722 INFO:tasks.ceph.mgr.x:Restarting daemon 2020-11-20T13:56:04.723 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2020-11-20T13:56:04.725 INFO:tasks.ceph.mgr.x:Started 2020-11-20T13:56:04.726 INFO:tasks.ceph:Setting crush tunables to default 2020-11-20T13:56:04.726 INFO:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph osd crush tunables default 2020-11-20T13:56:05.010 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:04.840+0000 7fea8975f700 1 Processor -- start 2020-11-20T13:56:05.010 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:04.840+0000 7fea8975f700 1 -- start start 2020-11-20T13:56:05.010 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:04.841+0000 7fea8975f700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fea840b3720 0x7fea84153e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:05.011 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:04.841+0000 7fea8975f700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea841574b0 con 0x7fea840805f0 2020-11-20T13:56:05.011 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:04.841+0000 7fea8975f700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea84157750 con 0x7fea84157bf0 2020-11-20T13:56:05.011 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:04.841+0000 7fea8975f700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea840809d0 con 0x7fea840b3720 2020-11-20T13:56:05.011 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:04.841+0000 7fea83fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fea840b3720 0x7fea84153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:05.012 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:04.841+0000 7fea837fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fea840805f0 0x7fea840b5ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44596/0 (socket says 172.21.15.40:44596) 2020-11-20T13:56:05.012 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:04.841+0000 7fea837fe700 1 -- 172.21.15.40:0/254027564 learned_addr learned my addr 172.21.15.40:0/254027564 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:05.012 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:04.841+0000 7fea83fff700 1 -- 172.21.15.40:0/254027564 >> v2:172.21.15.40:3300/0 conn(0x7fea840b3720 msgr2=0x7fea84153e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T13:56:05.012 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:04.841+0000 7fea83fff700 1 -- 172.21.15.40:0/254027564 >> v2:172.21.15.40:3300/0 conn(0x7fea840b3720 msgr2=0x7fea84153e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T13:56:05.012 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:04.841+0000 7fea83fff700 1 --2- 172.21.15.40:0/254027564 >> v2:172.21.15.40:3300/0 conn(0x7fea840b3720 0x7fea84153e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T13:56:05.013 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:04.841+0000 7fea83fff700 1 --2- 172.21.15.40:0/254027564 >> v2:172.21.15.40:3300/0 conn(0x7fea840b3720 0x7fea84153e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T13:56:05.013 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.005+0000 7fea827fc700 1 -- 172.21.15.40:0/254027564 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4210919231 0 0) 0x7fea841574b0 con 0x7fea840805f0 2020-11-20T13:56:05.013 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.005+0000 7fea827fc700 1 -- 172.21.15.40:0/254027564 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea6c003230 con 0x7fea840805f0 2020-11-20T13:56:05.013 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.005+0000 7fea827fc700 1 -- 172.21.15.40:0/254027564 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 777586791 0 0) 0x7fea84157750 con 0x7fea84157bf0 2020-11-20T13:56:05.014 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.005+0000 7fea827fc700 1 -- 172.21.15.40:0/254027564 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea841574b0 con 0x7fea84157bf0 2020-11-20T13:56:05.014 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.005+0000 7fea827fc700 1 -- 172.21.15.40:0/254027564 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3076460519 0 0) 0x7fea6c003230 con 0x7fea840805f0 2020-11-20T13:56:05.014 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.005+0000 7fea827fc700 1 -- 172.21.15.40:0/254027564 >> v2:172.21.15.40:3300/0 conn(0x7fea840b3720 msgr2=0x7fea84153e40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T13:56:05.014 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.005+0000 7fea827fc700 1 --2- 172.21.15.40:0/254027564 >> v2:172.21.15.40:3300/0 conn(0x7fea840b3720 0x7fea84153e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:05.014 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.005+0000 7fea827fc700 1 -- 172.21.15.40:0/254027564 >> v1:172.21.15.116:6789/0 conn(0x7fea84157bf0 legacy=0x7fea840b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:05.015 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.005+0000 7fea827fc700 1 -- 172.21.15.40:0/254027564 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea840b6820 con 0x7fea840805f0 2020-11-20T13:56:05.015 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.005+0000 7fea827fc700 1 -- 172.21.15.40:0/254027564 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2366654341 0 0) 0x7fea740016d0 con 0x7fea840805f0 2020-11-20T13:56:05.015 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.006+0000 7fea827fc700 1 -- 172.21.15.40:0/254027564 >> v1:172.21.15.40:6789/0 conn(0x7fea840805f0 legacy=0x7fea840b5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:05.015 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.006+0000 7fea827fc700 1 --2- 172.21.15.40:0/254027564 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea6c003900 0x7fea6c003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:05.016 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.006+0000 7fea827fc700 1 --2- 172.21.15.40:0/254027564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea6c0044d0 0x7fea6c004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:05.016 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.006+0000 7fea827fc700 1 --2- 172.21.15.40:0/254027564 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fea6c004e40 0x7fea6c04f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:05.016 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.006+0000 7fea827fc700 1 -- 172.21.15.40:0/254027564 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea840809d0 con 0x7fea6c0044d0 2020-11-20T13:56:05.017 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.006+0000 7fea827fc700 1 -- 172.21.15.40:0/254027564 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6c04fcb0 con 0x7fea6c003900 2020-11-20T13:56:05.017 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.006+0000 7fea827fc700 1 -- 172.21.15.40:0/254027564 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea6c04fe20 con 0x7fea6c004e40 2020-11-20T13:56:05.017 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.006+0000 7fea837fe700 1 --2- 172.21.15.40:0/254027564 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea6c003900 0x7fea6c003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:05.017 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.006+0000 7fea82ffd700 1 --2- 172.21.15.40:0/254027564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea6c0044d0 0x7fea6c004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:05.018 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.006+0000 7fea83fff700 1 --2- 172.21.15.40:0/254027564 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fea6c004e40 0x7fea6c04f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:05.018 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.006+0000 7fea82ffd700 1 -- 172.21.15.40:0/254027564 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fea6c004e40 msgr2=0x7fea6c04f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:05.018 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.006+0000 7fea82ffd700 1 --2- 172.21.15.40:0/254027564 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fea6c004e40 0x7fea6c04f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:05.018 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.006+0000 7fea82ffd700 1 -- 172.21.15.40:0/254027564 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea6c003900 msgr2=0x7fea6c003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:05.018 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.006+0000 7fea82ffd700 1 --2- 172.21.15.40:0/254027564 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea6c003900 0x7fea6c003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:05.019 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.006+0000 7fea82ffd700 1 -- 172.21.15.40:0/254027564 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea6c003230 con 0x7fea6c0044d0 2020-11-20T13:56:05.019 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.006+0000 7fea83fff700 1 -- 172.21.15.40:0/254027564 reap_dead start 2020-11-20T13:56:05.019 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.006+0000 7fea82ffd700 1 --2- 172.21.15.40:0/254027564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea6c0044d0 0x7fea6c004950 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fea780010d0 tx=0x7fea78007ad0).ready entity=mon.0 client_cookie=e735b278cfdcffa1 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:05.019 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.006+0000 7fea827fc700 1 -- 172.21.15.40:0/254027564 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea7800c680 con 0x7fea6c0044d0 2020-11-20T13:56:05.020 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.006+0000 7fea827fc700 1 -- 172.21.15.40:0/254027564 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea7800c7e0 con 0x7fea6c0044d0 2020-11-20T13:56:05.020 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.006+0000 7fea827fc700 1 -- 172.21.15.40:0/254027564 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea7800cb40 con 0x7fea6c0044d0 2020-11-20T13:56:05.020 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.006+0000 7fea8975f700 1 -- 172.21.15.40:0/254027564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea6c0044d0 msgr2=0x7fea6c004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:05.020 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.006+0000 7fea8975f700 1 --2- 172.21.15.40:0/254027564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea6c0044d0 0x7fea6c004950 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fea780010d0 tx=0x7fea78007ad0).stop 2020-11-20T13:56:05.021 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.007+0000 7fea8975f700 1 -- 172.21.15.40:0/254027564 shutdown_connections 2020-11-20T13:56:05.021 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.007+0000 7fea8975f700 1 --2- 172.21.15.40:0/254027564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea6c0044d0 0x7fea6c004950 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:05.021 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.007+0000 7fea8975f700 1 -- 172.21.15.40:0/254027564 >> 172.21.15.40:0/254027564 conn(0x7fea8405f390 msgr2=0x7fea8405f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:05.021 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.007+0000 7fea8975f700 1 -- 172.21.15.40:0/254027564 shutdown_connections 2020-11-20T13:56:05.022 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.007+0000 7fea8975f700 1 -- 172.21.15.40:0/254027564 wait complete. 2020-11-20T13:56:05.022 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.008+0000 7fea8975f700 1 Processor -- start 2020-11-20T13:56:05.022 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.009+0000 7fea8975f700 1 -- start start 2020-11-20T13:56:05.026 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.009+0000 7fea8975f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea6c0044d0 0x7fea84152430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:05.026 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.009+0000 7fea8975f700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fea840ae880 0x7fea8413f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:05.026 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.009+0000 7fea8975f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea84066a10 0x7fea84066e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:05.027 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.009+0000 7fea8975f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea84152920 con 0x7fea6c0044d0 2020-11-20T13:56:05.027 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.009+0000 7fea8975f700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8413f8b0 con 0x7fea84066a10 2020-11-20T13:56:05.027 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.009+0000 7fea8975f700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea8413e9e0 con 0x7fea840ae880 2020-11-20T13:56:05.028 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.009+0000 7fea82ffd700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fea840ae880 0x7fea8413f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:05.028 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.010+0000 7fea82ffd700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fea840ae880 0x7fea8413f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.40:39106/0 (socket says 172.21.15.40:39106) 2020-11-20T13:56:05.028 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.010+0000 7fea82ffd700 1 -- 172.21.15.40:0/3359980440 learned_addr learned my addr 172.21.15.40:0/3359980440 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:05.028 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.010+0000 7fea83fff700 1 --2- 172.21.15.40:0/3359980440 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea84066a10 0x7fea84066e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:05.028 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.010+0000 7fea837fe700 1 --2- 172.21.15.40:0/3359980440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea6c0044d0 0x7fea84152430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:05.029 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.010+0000 7fea82ffd700 1 -- 172.21.15.40:0/3359980440 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea84066a10 msgr2=0x7fea84066e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:05.029 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.010+0000 7fea82ffd700 1 --2- 172.21.15.40:0/3359980440 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea84066a10 0x7fea84066e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:05.029 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.010+0000 7fea82ffd700 1 -- 172.21.15.40:0/3359980440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea6c0044d0 msgr2=0x7fea84152430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:05.029 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.010+0000 7fea82ffd700 1 --2- 172.21.15.40:0/3359980440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea6c0044d0 0x7fea84152430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:05.030 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.010+0000 7fea82ffd700 1 -- 172.21.15.40:0/3359980440 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea8413ec80 con 0x7fea840ae880 2020-11-20T13:56:05.030 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.010+0000 7fea82ffd700 1 --2- 172.21.15.40:0/3359980440 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fea840ae880 0x7fea8413f3c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fea780010d0 tx=0x7fea78017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:05.030 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.014+0000 7fea6bfff700 1 -- 172.21.15.40:0/3359980440 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea78007800 con 0x7fea840ae880 2020-11-20T13:56:05.030 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.014+0000 7fea8975f700 1 -- 172.21.15.40:0/3359980440 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea8413ef20 con 0x7fea840ae880 2020-11-20T13:56:05.031 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.014+0000 7fea6bfff700 1 -- 172.21.15.40:0/3359980440 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea78007220 con 0x7fea840ae880 2020-11-20T13:56:05.031 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.014+0000 7fea6bfff700 1 -- 172.21.15.40:0/3359980440 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea7800ce60 con 0x7fea840ae880 2020-11-20T13:56:05.031 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.014+0000 7fea8975f700 1 -- 172.21.15.40:0/3359980440 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea840b3f90 con 0x7fea840ae880 2020-11-20T13:56:05.031 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.014+0000 7fea6bfff700 1 -- 172.21.15.40:0/3359980440 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7fea78010070 con 0x7fea840ae880 2020-11-20T13:56:05.032 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.014+0000 7fea6bfff700 1 -- 172.21.15.40:0/3359980440 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7fea780064e0 con 0x7fea840ae880 2020-11-20T13:56:05.032 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.015+0000 7fea8975f700 1 -- 172.21.15.40:0/3359980440 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea84057050 con 0x7fea840ae880 2020-11-20T13:56:05.032 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.021+0000 7fea6bfff700 1 -- 172.21.15.40:0/3359980440 <== mon.2 v2:172.21.15.116:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7fea78022070 con 0x7fea840ae880 2020-11-20T13:56:05.103 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.102+0000 7fea8975f700 1 -- 172.21.15.40:0/3359980440 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fea8406aca0 con 0x7fea840ae880 2020-11-20T13:56:05.107 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.106+0000 7fea6bfff700 1 -- 172.21.15.40:0/3359980440 <== mon.2 v2:172.21.15.116:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fea78001e60 con 0x7fea840ae880 2020-11-20T13:56:05.109 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.107+0000 7fea8975f700 1 -- 172.21.15.40:0/3359980440 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fea840ae880 msgr2=0x7fea8413f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:05.109 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.107+0000 7fea8975f700 1 --2- 172.21.15.40:0/3359980440 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fea840ae880 0x7fea8413f3c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fea780010d0 tx=0x7fea78017000).stop 2020-11-20T13:56:05.109 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.107+0000 7fea8975f700 1 -- 172.21.15.40:0/3359980440 shutdown_connections 2020-11-20T13:56:05.109 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.107+0000 7fea8975f700 1 --2- 172.21.15.40:0/3359980440 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea84066a10 0x7fea84066e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:05.110 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.107+0000 7fea8975f700 1 --2- 172.21.15.40:0/3359980440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea6c0044d0 0x7fea84152430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:05.110 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.107+0000 7fea8975f700 1 --2- 172.21.15.40:0/3359980440 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fea840ae880 0x7fea8413f3c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:05.110 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.107+0000 7fea8975f700 1 -- 172.21.15.40:0/3359980440 >> 172.21.15.40:0/3359980440 conn(0x7fea8405f390 msgr2=0x7fea840b7420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:05.110 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.107+0000 7fea8975f700 1 -- 172.21.15.40:0/3359980440 shutdown_connections 2020-11-20T13:56:05.110 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:05.107+0000 7fea8975f700 1 -- 172.21.15.40:0/3359980440 wait complete. 2020-11-20T13:56:05.111 INFO:teuthology.orchestra.run.smithi040.stderr:adjusted tunables profile to default 2020-11-20T13:56:05.119 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2020-11-20T13:56:05.120 INFO:teuthology.orchestra.run.smithi040:> mktemp 2020-11-20T13:56:05.134 INFO:teuthology.orchestra.run.smithi040.stdout:/tmp/tmp.WedlIUxo1O 2020-11-20T13:56:05.135 INFO:teuthology.orchestra.run.smithi040:> sudo cp /var/lib/ceph/osd/ceph-0/fsid /tmp/tmp.WedlIUxo1O 2020-11-20T13:56:05.206 INFO:teuthology.orchestra.run.smithi040:> sudo chmod 0666 /tmp/tmp.WedlIUxo1O 2020-11-20T13:56:05.334 DEBUG:teuthology.orchestra.remote:smithi040:/tmp/tmp.WedlIUxo1O is 37B 2020-11-20T13:56:05.382 INFO:teuthology.orchestra.run.smithi040:> rm -fr /tmp/tmp.WedlIUxo1O 2020-11-20T13:56:05.397 INFO:teuthology.orchestra.run.smithi040:> mktemp 2020-11-20T13:56:05.453 INFO:teuthology.orchestra.run.smithi040.stdout:/tmp/tmp.VV7255p6Dp 2020-11-20T13:56:05.453 INFO:teuthology.orchestra.run.smithi040:> sudo cp /var/lib/ceph/osd/ceph-1/fsid /tmp/tmp.VV7255p6Dp 2020-11-20T13:56:05.528 INFO:teuthology.orchestra.run.smithi040:> sudo chmod 0666 /tmp/tmp.VV7255p6Dp 2020-11-20T13:56:05.655 DEBUG:teuthology.orchestra.remote:smithi040:/tmp/tmp.VV7255p6Dp is 37B 2020-11-20T13:56:05.705 INFO:teuthology.orchestra.run.smithi040:> rm -fr /tmp/tmp.VV7255p6Dp 2020-11-20T13:56:05.721 INFO:teuthology.orchestra.run.smithi040:> mktemp 2020-11-20T13:56:05.774 INFO:teuthology.orchestra.run.smithi040.stdout:/tmp/tmp.Uf0A68A8lD 2020-11-20T13:56:05.775 INFO:teuthology.orchestra.run.smithi040:> sudo cp /var/lib/ceph/osd/ceph-2/fsid /tmp/tmp.Uf0A68A8lD 2020-11-20T13:56:05.847 INFO:teuthology.orchestra.run.smithi040:> sudo chmod 0666 /tmp/tmp.Uf0A68A8lD 2020-11-20T13:56:05.974 DEBUG:teuthology.orchestra.remote:smithi040:/tmp/tmp.Uf0A68A8lD is 37B 2020-11-20T13:56:06.028 INFO:teuthology.orchestra.run.smithi040:> rm -fr /tmp/tmp.Uf0A68A8lD 2020-11-20T13:56:06.042 INFO:teuthology.orchestra.run.smithi040:> mktemp 2020-11-20T13:56:06.107 INFO:teuthology.orchestra.run.smithi040.stdout:/tmp/tmp.T6QUimzKDE 2020-11-20T13:56:06.108 INFO:teuthology.orchestra.run.smithi040:> sudo cp /var/lib/ceph/osd/ceph-3/fsid /tmp/tmp.T6QUimzKDE 2020-11-20T13:56:06.187 INFO:teuthology.orchestra.run.smithi040:> sudo chmod 0666 /tmp/tmp.T6QUimzKDE 2020-11-20T13:56:06.315 DEBUG:teuthology.orchestra.remote:smithi040:/tmp/tmp.T6QUimzKDE is 37B 2020-11-20T13:56:06.364 INFO:teuthology.orchestra.run.smithi040:> rm -fr /tmp/tmp.T6QUimzKDE 2020-11-20T13:56:06.379 INFO:teuthology.orchestra.run.smithi116:> mktemp 2020-11-20T13:56:06.394 INFO:teuthology.orchestra.run.smithi116.stdout:/tmp/tmp.5f2rGA94gI 2020-11-20T13:56:06.395 INFO:teuthology.orchestra.run.smithi116:> sudo cp /var/lib/ceph/osd/ceph-4/fsid /tmp/tmp.5f2rGA94gI 2020-11-20T13:56:06.463 INFO:teuthology.orchestra.run.smithi116:> sudo chmod 0666 /tmp/tmp.5f2rGA94gI 2020-11-20T13:56:06.586 DEBUG:teuthology.orchestra.remote:smithi116:/tmp/tmp.5f2rGA94gI is 37B 2020-11-20T13:56:06.634 INFO:teuthology.orchestra.run.smithi116:> rm -fr /tmp/tmp.5f2rGA94gI 2020-11-20T13:56:06.648 INFO:teuthology.orchestra.run.smithi116:> mktemp 2020-11-20T13:56:06.702 INFO:teuthology.orchestra.run.smithi116.stdout:/tmp/tmp.LLP16DPGgj 2020-11-20T13:56:06.703 INFO:teuthology.orchestra.run.smithi116:> sudo cp /var/lib/ceph/osd/ceph-5/fsid /tmp/tmp.LLP16DPGgj 2020-11-20T13:56:06.769 INFO:teuthology.orchestra.run.smithi116:> sudo chmod 0666 /tmp/tmp.LLP16DPGgj 2020-11-20T13:56:06.895 DEBUG:teuthology.orchestra.remote:smithi116:/tmp/tmp.LLP16DPGgj is 37B 2020-11-20T13:56:06.943 INFO:teuthology.orchestra.run.smithi116:> rm -fr /tmp/tmp.LLP16DPGgj 2020-11-20T13:56:06.958 INFO:teuthology.orchestra.run.smithi116:> mktemp 2020-11-20T13:56:07.011 INFO:teuthology.orchestra.run.smithi116.stdout:/tmp/tmp.C55zidqOJB 2020-11-20T13:56:07.012 INFO:teuthology.orchestra.run.smithi116:> sudo cp /var/lib/ceph/osd/ceph-6/fsid /tmp/tmp.C55zidqOJB 2020-11-20T13:56:07.079 INFO:teuthology.orchestra.run.smithi116:> sudo chmod 0666 /tmp/tmp.C55zidqOJB 2020-11-20T13:56:07.204 DEBUG:teuthology.orchestra.remote:smithi116:/tmp/tmp.C55zidqOJB is 37B 2020-11-20T13:56:07.255 INFO:teuthology.orchestra.run.smithi116:> rm -fr /tmp/tmp.C55zidqOJB 2020-11-20T13:56:07.270 INFO:teuthology.orchestra.run.smithi116:> mktemp 2020-11-20T13:56:07.323 INFO:teuthology.orchestra.run.smithi116.stdout:/tmp/tmp.MGibJ5eF5R 2020-11-20T13:56:07.324 INFO:teuthology.orchestra.run.smithi116:> sudo cp /var/lib/ceph/osd/ceph-7/fsid /tmp/tmp.MGibJ5eF5R 2020-11-20T13:56:07.391 INFO:teuthology.orchestra.run.smithi116:> sudo chmod 0666 /tmp/tmp.MGibJ5eF5R 2020-11-20T13:56:07.514 DEBUG:teuthology.orchestra.remote:smithi116:/tmp/tmp.MGibJ5eF5R is 37B 2020-11-20T13:56:07.562 INFO:teuthology.orchestra.run.smithi116:> rm -fr /tmp/tmp.MGibJ5eF5R 2020-11-20T13:56:07.576 INFO:teuthology.orchestra.run.smithi116:> sudo ceph --cluster ceph osd new 9ea4c667-8938-46d6-a5ca-092d55c1f79e 0 2020-11-20T13:56:07.725 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.720+0000 7fbf1ce81700 1 Processor -- start 2020-11-20T13:56:07.725 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.721+0000 7fbf1ce81700 1 -- start start 2020-11-20T13:56:07.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.721+0000 7fbf1ce81700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbf18157bf0 0x7fbf180b3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:07.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.721+0000 7fbf1ce81700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf181574b0 con 0x7fbf18158070 2020-11-20T13:56:07.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.721+0000 7fbf1ce81700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf18157750 con 0x7fbf180805f0 2020-11-20T13:56:07.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.721+0000 7fbf1ce81700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf180809d0 con 0x7fbf18157bf0 2020-11-20T13:56:07.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.722+0000 7fbf16d9d700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fbf180805f0 0x7fbf180b5ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:50532/0 (socket says 172.21.15.116:50532) 2020-11-20T13:56:07.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.722+0000 7fbf16d9d700 1 -- 172.21.15.116:0/1216744899 learned_addr learned my addr 172.21.15.116:0/1216744899 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T13:56:07.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.722+0000 7fbf1659c700 1 --2- 172.21.15.116:0/1216744899 >> v2:172.21.15.40:3300/0 conn(0x7fbf18157bf0 0x7fbf180b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:07.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.722+0000 7fbf15d9b700 1 -- 172.21.15.116:0/1216744899 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3084620923 0 0) 0x7fbf18157750 con 0x7fbf180805f0 2020-11-20T13:56:07.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.722+0000 7fbf15d9b700 1 -- 172.21.15.116:0/1216744899 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf00003230 con 0x7fbf180805f0 2020-11-20T13:56:07.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.722+0000 7fbf15d9b700 1 -- 172.21.15.116:0/1216744899 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3728168704 0 0) 0x7fbf181574b0 con 0x7fbf18158070 2020-11-20T13:56:07.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.722+0000 7fbf15d9b700 1 -- 172.21.15.116:0/1216744899 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf18157750 con 0x7fbf18158070 2020-11-20T13:56:07.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.722+0000 7fbf1659c700 1 -- 172.21.15.116:0/1216744899 >> v1:172.21.15.116:6789/0 conn(0x7fbf180805f0 legacy=0x7fbf180b5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:07.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.722+0000 7fbf1659c700 1 -- 172.21.15.116:0/1216744899 >> v1:172.21.15.40:6789/0 conn(0x7fbf18158070 legacy=0x7fbf18153e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:07.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.722+0000 7fbf1659c700 1 -- 172.21.15.116:0/1216744899 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf180b6820 con 0x7fbf18157bf0 2020-11-20T13:56:07.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.722+0000 7fbf1659c700 1 --2- 172.21.15.116:0/1216744899 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf18157bf0 0x7fbf180b3010 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fbf08002f30 tx=0x7fbf08006dc0).ready entity=mon.0 client_cookie=8d34ff45eff4f89f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:07.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.722+0000 7fbf15d9b700 1 -- 172.21.15.116:0/1216744899 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf08009740 con 0x7fbf18157bf0 2020-11-20T13:56:07.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.722+0000 7fbf15d9b700 1 -- 172.21.15.116:0/1216744899 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf080098a0 con 0x7fbf18157bf0 2020-11-20T13:56:07.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.722+0000 7fbf15d9b700 1 -- 172.21.15.116:0/1216744899 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf08009c00 con 0x7fbf18157bf0 2020-11-20T13:56:07.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.722+0000 7fbf1ce81700 1 -- 172.21.15.116:0/1216744899 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf18157bf0 msgr2=0x7fbf180b3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:07.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.722+0000 7fbf1ce81700 1 --2- 172.21.15.116:0/1216744899 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf18157bf0 0x7fbf180b3010 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fbf08002f30 tx=0x7fbf08006dc0).stop 2020-11-20T13:56:07.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.723+0000 7fbf1ce81700 1 -- 172.21.15.116:0/1216744899 shutdown_connections 2020-11-20T13:56:07.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.723+0000 7fbf1ce81700 1 --2- 172.21.15.116:0/1216744899 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf18157bf0 0x7fbf180b3010 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:07.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.723+0000 7fbf1ce81700 1 -- 172.21.15.116:0/1216744899 >> 172.21.15.116:0/1216744899 conn(0x7fbf1805f390 msgr2=0x7fbf1805f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:07.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.723+0000 7fbf1ce81700 1 -- 172.21.15.116:0/1216744899 shutdown_connections 2020-11-20T13:56:07.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.723+0000 7fbf1ce81700 1 -- 172.21.15.116:0/1216744899 wait complete. 2020-11-20T13:56:07.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.724+0000 7fbf1ce81700 1 Processor -- start 2020-11-20T13:56:07.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.724+0000 7fbf1ce81700 1 -- start start 2020-11-20T13:56:07.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.725+0000 7fbf1ce81700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf180805f0 0x7fbf18150ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:07.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.725+0000 7fbf1ce81700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbf18157bf0 0x7fbf1812c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:07.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.725+0000 7fbf1ce81700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf18158070 0x7fbf1812d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:07.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.725+0000 7fbf1ce81700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1812cec0 con 0x7fbf180805f0 2020-11-20T13:56:07.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.725+0000 7fbf1ce81700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1812d160 con 0x7fbf18158070 2020-11-20T13:56:07.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.725+0000 7fbf1ce81700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1812d400 con 0x7fbf18157bf0 2020-11-20T13:56:07.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.725+0000 7fbf1659c700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbf18157bf0 0x7fbf1812c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:07.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.725+0000 7fbf1759e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf18158070 0x7fbf1812d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:07.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.725+0000 7fbf1659c700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbf18157bf0 0x7fbf1812c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.116:40468/0 (socket says 172.21.15.116:40468) 2020-11-20T13:56:07.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.725+0000 7fbf1659c700 1 -- 172.21.15.116:0/1894091201 learned_addr learned my addr 172.21.15.116:0/1894091201 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T13:56:07.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.725+0000 7fbf16d9d700 1 --2- 172.21.15.116:0/1894091201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf180805f0 0x7fbf18150ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:07.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.725+0000 7fbf1759e700 1 -- 172.21.15.116:0/1894091201 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbf18157bf0 msgr2=0x7fbf1812c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:07.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.725+0000 7fbf1759e700 1 --2- 172.21.15.116:0/1894091201 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbf18157bf0 0x7fbf1812c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:07.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.725+0000 7fbf1759e700 1 -- 172.21.15.116:0/1894091201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf180805f0 msgr2=0x7fbf18150ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:07.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.725+0000 7fbf1759e700 1 --2- 172.21.15.116:0/1894091201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf180805f0 0x7fbf18150ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:07.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.725+0000 7fbf1759e700 1 -- 172.21.15.116:0/1894091201 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf1813c890 con 0x7fbf18158070 2020-11-20T13:56:07.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.725+0000 7fbf1659c700 1 --2- 172.21.15.116:0/1894091201 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbf18157bf0 0x7fbf1812c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:56:07.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.725+0000 7fbf1759e700 1 --2- 172.21.15.116:0/1894091201 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf18158070 0x7fbf1812d8a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fbf10001450 tx=0x7fbf10007c80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:07.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.725+0000 7fbf077fe700 1 -- 172.21.15.116:0/1894091201 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf1000a5a0 con 0x7fbf18158070 2020-11-20T13:56:07.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.725+0000 7fbf077fe700 1 -- 172.21.15.116:0/1894091201 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf1000a700 con 0x7fbf18158070 2020-11-20T13:56:07.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.725+0000 7fbf1ce81700 1 -- 172.21.15.116:0/1894091201 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf1812caa0 con 0x7fbf18158070 2020-11-20T13:56:07.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.725+0000 7fbf077fe700 1 -- 172.21.15.116:0/1894091201 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf1000a870 con 0x7fbf18158070 2020-11-20T13:56:07.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.726+0000 7fbf1ce81700 1 -- 172.21.15.116:0/1894091201 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf1812df50 con 0x7fbf18158070 2020-11-20T13:56:07.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.726+0000 7fbf077fe700 1 -- 172.21.15.116:0/1894091201 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7fbf10010070 con 0x7fbf18158070 2020-11-20T13:56:07.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.726+0000 7fbf077fe700 1 -- 172.21.15.116:0/1894091201 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 754+0+0 (secure 0 0 0) 0x7fbf1001b770 con 0x7fbf18158070 2020-11-20T13:56:07.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.727+0000 7fbf1ce81700 1 -- 172.21.15.116:0/1894091201 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf1812d5c0 con 0x7fbf18158070 2020-11-20T13:56:07.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.729+0000 7fbf077fe700 1 -- 172.21.15.116:0/1894091201 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7fbf1001b980 con 0x7fbf18158070 2020-11-20T13:56:07.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.815+0000 7fbf1ce81700 1 -- 172.21.15.116:0/1894091201 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9ea4c667-8938-46d6-a5ca-092d55c1f79e", "id": 0} v 0) v1 -- 0x7fbf18057050 con 0x7fbf18158070 2020-11-20T13:56:07.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.818+0000 7fbf077fe700 1 -- 172.21.15.116:0/1894091201 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9ea4c667-8938-46d6-a5ca-092d55c1f79e", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbf1002a0b0 con 0x7fbf18158070 2020-11-20T13:56:07.820 INFO:teuthology.orchestra.run.smithi116.stdout:0 2020-11-20T13:56:07.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.819+0000 7fbf1ce81700 1 -- 172.21.15.116:0/1894091201 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf18158070 msgr2=0x7fbf1812d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:07.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.819+0000 7fbf1ce81700 1 --2- 172.21.15.116:0/1894091201 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf18158070 0x7fbf1812d8a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fbf10001450 tx=0x7fbf10007c80).stop 2020-11-20T13:56:07.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.820+0000 7fbf1ce81700 1 -- 172.21.15.116:0/1894091201 shutdown_connections 2020-11-20T13:56:07.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.820+0000 7fbf1ce81700 1 --2- 172.21.15.116:0/1894091201 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf18158070 0x7fbf1812d8a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:07.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.820+0000 7fbf1ce81700 1 --2- 172.21.15.116:0/1894091201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf180805f0 0x7fbf18150ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:07.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.820+0000 7fbf1ce81700 1 --2- 172.21.15.116:0/1894091201 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbf18157bf0 0x7fbf1812c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:07.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.820+0000 7fbf1ce81700 1 -- 172.21.15.116:0/1894091201 >> 172.21.15.116:0/1894091201 conn(0x7fbf1805f390 msgr2=0x7fbf180b2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:07.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.820+0000 7fbf1ce81700 1 -- 172.21.15.116:0/1894091201 shutdown_connections 2020-11-20T13:56:07.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.820+0000 7fbf1ce81700 1 -- 172.21.15.116:0/1894091201 wait complete. 2020-11-20T13:56:07.831 INFO:teuthology.orchestra.run.smithi116:> sudo ceph --cluster ceph osd new 515708d9-5087-4e55-8d14-cdea1c00eacc 1 2020-11-20T13:56:07.943 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.939+0000 7f257f3fc700 1 Processor -- start 2020-11-20T13:56:07.943 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.939+0000 7f257f3fc700 1 -- start start 2020-11-20T13:56:07.944 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.940+0000 7f257f3fc700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f25780805f0 0x7f25780b5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:07.944 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.940+0000 7f257f3fc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f25780b6390 0x7f2578157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:07.944 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.940+0000 7f257f3fc700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2578157750 con 0x7f25781580e0 2020-11-20T13:56:07.944 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.940+0000 7f257f3fc700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25780809d0 con 0x7f25780b6390 2020-11-20T13:56:07.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.940+0000 7f257f3fc700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25780b6820 con 0x7f25780805f0 2020-11-20T13:56:07.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.940+0000 7f257d999700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f25780805f0 0x7f25780b5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:07.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.940+0000 7f257d999700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f25780805f0 0x7f25780b5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.116:40472/0 (socket says 172.21.15.116:40472) 2020-11-20T13:56:07.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.940+0000 7f257d999700 1 -- 172.21.15.116:0/1160246706 learned_addr learned my addr 172.21.15.116:0/1160246706 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T13:56:07.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.940+0000 7f257d198700 1 --2- 172.21.15.116:0/1160246706 >> v2:172.21.15.40:3300/0 conn(0x7f25780b6390 0x7f2578157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:07.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.940+0000 7f257d999700 1 -- 172.21.15.116:0/1160246706 >> v2:172.21.15.40:3300/0 conn(0x7f25780b6390 msgr2=0x7f2578157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:07.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.940+0000 7f257d999700 1 --2- 172.21.15.116:0/1160246706 >> v2:172.21.15.40:3300/0 conn(0x7f25780b6390 0x7f2578157bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:07.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.940+0000 7f257d999700 1 -- 172.21.15.116:0/1160246706 >> v1:172.21.15.40:6789/0 conn(0x7f25781580e0 legacy=0x7f25780b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:07.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.940+0000 7f257d999700 1 -- 172.21.15.116:0/1160246706 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25780b18c0 con 0x7f25780805f0 2020-11-20T13:56:07.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.940+0000 7f257d999700 1 --2- 172.21.15.116:0/1160246706 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f25780805f0 0x7f25780b5ea0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f256c002f30 tx=0x7f256c006e80).ready entity=mon.2 client_cookie=20c46b427baa6bfc server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:07.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.940+0000 7f257c997700 1 -- 172.21.15.116:0/1160246706 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f256c0097f0 con 0x7f25780805f0 2020-11-20T13:56:07.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.941+0000 7f257c997700 1 -- 172.21.15.116:0/1160246706 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f256c009950 con 0x7f25780805f0 2020-11-20T13:56:07.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.941+0000 7f257c997700 1 -- 172.21.15.116:0/1160246706 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f256c009cb0 con 0x7f25780805f0 2020-11-20T13:56:07.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.941+0000 7f257f3fc700 1 -- 172.21.15.116:0/1160246706 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f25780805f0 msgr2=0x7f25780b5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:07.948 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.941+0000 7f257f3fc700 1 --2- 172.21.15.116:0/1160246706 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f25780805f0 0x7f25780b5ea0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f256c002f30 tx=0x7f256c006e80).stop 2020-11-20T13:56:07.948 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.941+0000 7f257f3fc700 1 -- 172.21.15.116:0/1160246706 shutdown_connections 2020-11-20T13:56:07.948 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.941+0000 7f257f3fc700 1 --2- 172.21.15.116:0/1160246706 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f25780805f0 0x7f25780b5ea0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:07.948 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.941+0000 7f257f3fc700 1 --2- 172.21.15.116:0/1160246706 >> v2:172.21.15.40:3300/0 conn(0x7f25780b6390 0x7f2578157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:07.948 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.941+0000 7f257f3fc700 1 -- 172.21.15.116:0/1160246706 >> 172.21.15.116:0/1160246706 conn(0x7f257805f390 msgr2=0x7f257805f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:07.949 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.941+0000 7f257f3fc700 1 -- 172.21.15.116:0/1160246706 shutdown_connections 2020-11-20T13:56:07.949 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.941+0000 7f257f3fc700 1 -- 172.21.15.116:0/1160246706 wait complete. 2020-11-20T13:56:07.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.942+0000 7f257f3fc700 1 Processor -- start 2020-11-20T13:56:07.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.942+0000 7f257f3fc700 1 -- start start 2020-11-20T13:56:07.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.943+0000 7f257f3fc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25780805f0 0x7f2578138600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:07.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.943+0000 7f257f3fc700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f25781580e0 0x7f2578152730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:07.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.943+0000 7f257f3fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2578066a10 0x7f2578153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:07.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.943+0000 7f257f3fc700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257814e380 con 0x7f2578066a10 2020-11-20T13:56:07.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.943+0000 7f257f3fc700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25780afad0 con 0x7f25780805f0 2020-11-20T13:56:07.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.943+0000 7f257f3fc700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25780afd70 con 0x7f25781580e0 2020-11-20T13:56:07.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.943+0000 7f257d198700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f25781580e0 0x7f2578152730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:07.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.943+0000 7f257d999700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25780805f0 0x7f2578138600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:07.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.943+0000 7f257d198700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f25781580e0 0x7f2578152730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.116:40480/0 (socket says 172.21.15.116:40480) 2020-11-20T13:56:07.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.943+0000 7f257d198700 1 -- 172.21.15.116:0/3391357847 learned_addr learned my addr 172.21.15.116:0/3391357847 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T13:56:07.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.943+0000 7f257e19a700 1 --2- 172.21.15.116:0/3391357847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2578066a10 0x7f2578153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:07.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.943+0000 7f257d999700 1 -- 172.21.15.116:0/3391357847 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f25781580e0 msgr2=0x7f2578152730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:07.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.943+0000 7f257d999700 1 --2- 172.21.15.116:0/3391357847 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f25781580e0 0x7f2578152730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:07.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.943+0000 7f257d999700 1 -- 172.21.15.116:0/3391357847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2578066a10 msgr2=0x7f2578153e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:07.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.943+0000 7f257d999700 1 --2- 172.21.15.116:0/3391357847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2578066a10 0x7f2578153e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:07.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.943+0000 7f257d999700 1 -- 172.21.15.116:0/3391357847 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2578152c20 con 0x7f25780805f0 2020-11-20T13:56:07.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.943+0000 7f257d198700 1 --2- 172.21.15.116:0/3391357847 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f25781580e0 0x7f2578152730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:56:07.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.944+0000 7f257d999700 1 --2- 172.21.15.116:0/3391357847 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25780805f0 0x7f2578138600 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f256c002f30 tx=0x7f256c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:07.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.944+0000 7f25767fc700 1 -- 172.21.15.116:0/3391357847 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f256c011b00 con 0x7f25780805f0 2020-11-20T13:56:07.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.944+0000 7f25767fc700 1 -- 172.21.15.116:0/3391357847 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f256c011c60 con 0x7f25780805f0 2020-11-20T13:56:07.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.944+0000 7f257f3fc700 1 -- 172.21.15.116:0/3391357847 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25780add90 con 0x7f25780805f0 2020-11-20T13:56:07.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.944+0000 7f25767fc700 1 -- 172.21.15.116:0/3391357847 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f256c01e460 con 0x7f25780805f0 2020-11-20T13:56:07.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.944+0000 7f257f3fc700 1 -- 172.21.15.116:0/3391357847 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25781521d0 con 0x7f25780805f0 2020-11-20T13:56:07.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.944+0000 7f25767fc700 1 -- 172.21.15.116:0/3391357847 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f256c025030 con 0x7f25780805f0 2020-11-20T13:56:07.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.944+0000 7f25767fc700 1 -- 172.21.15.116:0/3391357847 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 869+0+0 (secure 0 0 0) 0x7f256c01ec70 con 0x7f25780805f0 2020-11-20T13:56:07.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.945+0000 7f257f3fc700 1 -- 172.21.15.116:0/3391357847 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2578057050 con 0x7f25780805f0 2020-11-20T13:56:07.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:07.947+0000 7f25767fc700 1 -- 172.21.15.116:0/3391357847 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f256c023070 con 0x7f25780805f0 2020-11-20T13:56:08.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.032+0000 7f257f3fc700 1 -- 172.21.15.116:0/3391357847 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd new", "uuid": "515708d9-5087-4e55-8d14-cdea1c00eacc", "id": 1} v 0) v1 -- 0x7f25781571f0 con 0x7f25780805f0 2020-11-20T13:56:08.039 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.037+0000 7f25767fc700 1 -- 172.21.15.116:0/3391357847 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "515708d9-5087-4e55-8d14-cdea1c00eacc", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f256c031480 con 0x7f25780805f0 2020-11-20T13:56:08.039 INFO:teuthology.orchestra.run.smithi116.stdout:1 2020-11-20T13:56:08.040 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.038+0000 7f257f3fc700 1 -- 172.21.15.116:0/3391357847 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25780805f0 msgr2=0x7f2578138600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:08.041 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.038+0000 7f257f3fc700 1 --2- 172.21.15.116:0/3391357847 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25780805f0 0x7f2578138600 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f256c002f30 tx=0x7f256c015000).stop 2020-11-20T13:56:08.041 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.038+0000 7f257f3fc700 1 -- 172.21.15.116:0/3391357847 shutdown_connections 2020-11-20T13:56:08.041 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.038+0000 7f257f3fc700 1 --2- 172.21.15.116:0/3391357847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2578066a10 0x7f2578153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.041 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.038+0000 7f257f3fc700 1 --2- 172.21.15.116:0/3391357847 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25780805f0 0x7f2578138600 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.041 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.038+0000 7f257f3fc700 1 --2- 172.21.15.116:0/3391357847 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f25781580e0 0x7f2578152730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.042 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.038+0000 7f257f3fc700 1 -- 172.21.15.116:0/3391357847 >> 172.21.15.116:0/3391357847 conn(0x7f257805f390 msgr2=0x7f25780b8180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:08.042 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.039+0000 7f257f3fc700 1 -- 172.21.15.116:0/3391357847 shutdown_connections 2020-11-20T13:56:08.042 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.039+0000 7f257f3fc700 1 -- 172.21.15.116:0/3391357847 wait complete. 2020-11-20T13:56:08.050 INFO:teuthology.orchestra.run.smithi116:> sudo ceph --cluster ceph osd new 4d86bca7-743e-4983-b35f-36676c405b36 2 2020-11-20T13:56:08.160 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.156+0000 7fe3835d2700 1 Processor -- start 2020-11-20T13:56:08.160 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.156+0000 7fe3835d2700 1 -- start start 2020-11-20T13:56:08.160 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.156+0000 7fe3835d2700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe37c0805f0 0x7fe37c0b5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:08.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.156+0000 7fe3835d2700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe37c0b3010 0x7fe37c0b3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:08.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.156+0000 7fe3835d2700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe37c157750 con 0x7fe37c0b6390 2020-11-20T13:56:08.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.156+0000 7fe3835d2700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe37c157540 con 0x7fe37c0b3010 2020-11-20T13:56:08.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.156+0000 7fe3835d2700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe37c0809d0 con 0x7fe37c0805f0 2020-11-20T13:56:08.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.157+0000 7fe381b6f700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe37c0805f0 0x7fe37c0b5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:08.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.157+0000 7fe381b6f700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe37c0805f0 0x7fe37c0b5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48104/0 (socket says 172.21.15.116:48104) 2020-11-20T13:56:08.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.157+0000 7fe381b6f700 1 -- 172.21.15.116:0/1041538654 learned_addr learned my addr 172.21.15.116:0/1041538654 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T13:56:08.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.157+0000 7fe382370700 1 --2- 172.21.15.116:0/1041538654 >> v2:172.21.15.40:3300/0 conn(0x7fe37c0b3010 0x7fe37c0b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:08.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.157+0000 7fe381b6f700 1 -- 172.21.15.116:0/1041538654 >> v2:172.21.15.40:3300/0 conn(0x7fe37c0b3010 msgr2=0x7fe37c0b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:08.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.157+0000 7fe381b6f700 1 --2- 172.21.15.116:0/1041538654 >> v2:172.21.15.40:3300/0 conn(0x7fe37c0b3010 0x7fe37c0b3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.163 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.157+0000 7fe381b6f700 1 -- 172.21.15.116:0/1041538654 >> v1:172.21.15.40:6789/0 conn(0x7fe37c0b6390 legacy=0x7fe37c157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:08.163 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.157+0000 7fe381b6f700 1 -- 172.21.15.116:0/1041538654 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe37c158300 con 0x7fe37c0805f0 2020-11-20T13:56:08.163 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.157+0000 7fe381b6f700 1 --2- 172.21.15.116:0/1041538654 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe37c0805f0 0x7fe37c0b5ea0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fe378002f30 tx=0x7fe378006e80).ready entity=mon.1 client_cookie=b1ea4f715464ceff server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:08.163 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.157+0000 7fe380b6d700 1 -- 172.21.15.116:0/1041538654 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3780097f0 con 0x7fe37c0805f0 2020-11-20T13:56:08.163 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.157+0000 7fe380b6d700 1 -- 172.21.15.116:0/1041538654 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe378009950 con 0x7fe37c0805f0 2020-11-20T13:56:08.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.157+0000 7fe380b6d700 1 -- 172.21.15.116:0/1041538654 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe378009cb0 con 0x7fe37c0805f0 2020-11-20T13:56:08.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.157+0000 7fe3835d2700 1 -- 172.21.15.116:0/1041538654 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe37c0805f0 msgr2=0x7fe37c0b5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:08.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.157+0000 7fe3835d2700 1 --2- 172.21.15.116:0/1041538654 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe37c0805f0 0x7fe37c0b5ea0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fe378002f30 tx=0x7fe378006e80).stop 2020-11-20T13:56:08.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.158+0000 7fe3835d2700 1 -- 172.21.15.116:0/1041538654 shutdown_connections 2020-11-20T13:56:08.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.158+0000 7fe3835d2700 1 --2- 172.21.15.116:0/1041538654 >> v2:172.21.15.40:3300/0 conn(0x7fe37c0b3010 0x7fe37c0b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.158+0000 7fe3835d2700 1 --2- 172.21.15.116:0/1041538654 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe37c0805f0 0x7fe37c0b5ea0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.158+0000 7fe3835d2700 1 -- 172.21.15.116:0/1041538654 >> 172.21.15.116:0/1041538654 conn(0x7fe37c05f390 msgr2=0x7fe37c05f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:08.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.158+0000 7fe3835d2700 1 -- 172.21.15.116:0/1041538654 shutdown_connections 2020-11-20T13:56:08.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.158+0000 7fe3835d2700 1 -- 172.21.15.116:0/1041538654 wait complete. 2020-11-20T13:56:08.168 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.159+0000 7fe3835d2700 1 Processor -- start 2020-11-20T13:56:08.168 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.159+0000 7fe3835d2700 1 -- start start 2020-11-20T13:56:08.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.159+0000 7fe3835d2700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe37c0805f0 0x7fe37c063f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:08.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.160+0000 7fe3835d2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe37c0b3010 0x7fe37c0b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:08.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.160+0000 7fe3835d2700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe37c0b0a90 0x7fe37c1213b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:08.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.160+0000 7fe3835d2700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c0b0f20 con 0x7fe37c0b3010 2020-11-20T13:56:08.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.160+0000 7fe3835d2700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c120c70 con 0x7fe37c0805f0 2020-11-20T13:56:08.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.160+0000 7fe3835d2700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c120f10 con 0x7fe37c0b0a90 2020-11-20T13:56:08.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.160+0000 7fe381b6f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe37c0805f0 0x7fe37c063f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:08.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.160+0000 7fe381b6f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe37c0805f0 0x7fe37c063f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48110/0 (socket says 172.21.15.116:48110) 2020-11-20T13:56:08.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.160+0000 7fe381b6f700 1 -- 172.21.15.116:0/3603902788 learned_addr learned my addr 172.21.15.116:0/3603902788 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T13:56:08.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.160+0000 7fe382370700 1 --2- 172.21.15.116:0/3603902788 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe37c0b0a90 0x7fe37c1213b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:08.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.160+0000 7fe38136e700 1 --2- 172.21.15.116:0/3603902788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe37c0b3010 0x7fe37c0b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:08.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.160+0000 7fe381b6f700 1 -- 172.21.15.116:0/3603902788 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe37c0b0a90 msgr2=0x7fe37c1213b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:08.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.160+0000 7fe381b6f700 1 --2- 172.21.15.116:0/3603902788 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe37c0b0a90 0x7fe37c1213b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.160+0000 7fe381b6f700 1 -- 172.21.15.116:0/3603902788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe37c0b3010 msgr2=0x7fe37c0b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:08.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.160+0000 7fe381b6f700 1 --2- 172.21.15.116:0/3603902788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe37c0b3010 0x7fe37c0b05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.160+0000 7fe381b6f700 1 -- 172.21.15.116:0/3603902788 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe37c067400 con 0x7fe37c0805f0 2020-11-20T13:56:08.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.160+0000 7fe381b6f700 1 --2- 172.21.15.116:0/3603902788 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe37c0805f0 0x7fe37c063f40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fe378002f30 tx=0x7fe378015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:08.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.160+0000 7fe36e7fc700 1 -- 172.21.15.116:0/3603902788 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe378011b80 con 0x7fe37c0805f0 2020-11-20T13:56:08.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.160+0000 7fe36e7fc700 1 -- 172.21.15.116:0/3603902788 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe378011ce0 con 0x7fe37c0805f0 2020-11-20T13:56:08.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.160+0000 7fe3835d2700 1 -- 172.21.15.116:0/3603902788 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe37c1218a0 con 0x7fe37c0805f0 2020-11-20T13:56:08.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.160+0000 7fe36e7fc700 1 -- 172.21.15.116:0/3603902788 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe37801e650 con 0x7fe37c0805f0 2020-11-20T13:56:08.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.160+0000 7fe3835d2700 1 -- 172.21.15.116:0/3603902788 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe37c121b50 con 0x7fe37c0805f0 2020-11-20T13:56:08.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.161+0000 7fe36e7fc700 1 -- 172.21.15.116:0/3603902788 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7fe378025030 con 0x7fe37c0805f0 2020-11-20T13:56:08.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.161+0000 7fe36e7fc700 1 -- 172.21.15.116:0/3603902788 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 984+0+0 (secure 0 0 0) 0x7fe37801d810 con 0x7fe37c0805f0 2020-11-20T13:56:08.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.161+0000 7fe3835d2700 1 -- 172.21.15.116:0/3603902788 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe37c0b1090 con 0x7fe37c0805f0 2020-11-20T13:56:08.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.163+0000 7fe36e7fc700 1 -- 172.21.15.116:0/3603902788 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7fe37c0b1090 con 0x7fe37c0805f0 2020-11-20T13:56:08.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.252+0000 7fe3835d2700 1 -- 172.21.15.116:0/3603902788 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4d86bca7-743e-4983-b35f-36676c405b36", "id": 2} v 0) v1 -- 0x7fe37c1210d0 con 0x7fe37c0805f0 2020-11-20T13:56:08.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.259+0000 7fe36e7fc700 1 -- 172.21.15.116:0/3603902788 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4d86bca7-743e-4983-b35f-36676c405b36", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe378023070 con 0x7fe37c0805f0 2020-11-20T13:56:08.261 INFO:teuthology.orchestra.run.smithi116.stdout:2 2020-11-20T13:56:08.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.260+0000 7fe3835d2700 1 -- 172.21.15.116:0/3603902788 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe37c0805f0 msgr2=0x7fe37c063f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:08.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.260+0000 7fe3835d2700 1 --2- 172.21.15.116:0/3603902788 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe37c0805f0 0x7fe37c063f40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fe378002f30 tx=0x7fe378015000).stop 2020-11-20T13:56:08.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.260+0000 7fe3835d2700 1 -- 172.21.15.116:0/3603902788 shutdown_connections 2020-11-20T13:56:08.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.260+0000 7fe3835d2700 1 --2- 172.21.15.116:0/3603902788 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe37c0b0a90 0x7fe37c1213b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.260+0000 7fe3835d2700 1 --2- 172.21.15.116:0/3603902788 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe37c0805f0 0x7fe37c063f40 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.260+0000 7fe3835d2700 1 --2- 172.21.15.116:0/3603902788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe37c0b3010 0x7fe37c0b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.260+0000 7fe3835d2700 1 -- 172.21.15.116:0/3603902788 >> 172.21.15.116:0/3603902788 conn(0x7fe37c05f390 msgr2=0x7fe37c0b7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:08.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.260+0000 7fe3835d2700 1 -- 172.21.15.116:0/3603902788 shutdown_connections 2020-11-20T13:56:08.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.260+0000 7fe3835d2700 1 -- 172.21.15.116:0/3603902788 wait complete. 2020-11-20T13:56:08.271 INFO:teuthology.orchestra.run.smithi116:> sudo ceph --cluster ceph osd new f4dc726b-6b34-4e8b-8eeb-87060b58002d 3 2020-11-20T13:56:08.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.380+0000 7efca533b700 1 Processor -- start 2020-11-20T13:56:08.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.380+0000 7efca533b700 1 -- start start 2020-11-20T13:56:08.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.381+0000 7efca533b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efca00805f0 0x7efca00b5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:08.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.381+0000 7efca533b700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efca00b3010 0x7efca00b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:08.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.381+0000 7efca533b700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efca0157750 con 0x7efca00b6390 2020-11-20T13:56:08.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.381+0000 7efca533b700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efca0157570 con 0x7efca00805f0 2020-11-20T13:56:08.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.381+0000 7efca533b700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efca00809d0 con 0x7efca00b3010 2020-11-20T13:56:08.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.381+0000 7efc9ffff700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efca00b3010 0x7efca00b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:08.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.381+0000 7efc9ffff700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efca00b3010 0x7efca00b34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.116:40526/0 (socket says 172.21.15.116:40526) 2020-11-20T13:56:08.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.381+0000 7efc9ffff700 1 -- 172.21.15.116:0/2819847668 learned_addr learned my addr 172.21.15.116:0/2819847668 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T13:56:08.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.381+0000 7efc9f7fe700 1 --2- 172.21.15.116:0/2819847668 >> v2:172.21.15.40:3300/0 conn(0x7efca00805f0 0x7efca00b5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:08.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.381+0000 7efc9ffff700 1 -- 172.21.15.116:0/2819847668 >> v2:172.21.15.40:3300/0 conn(0x7efca00805f0 msgr2=0x7efca00b5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:08.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.381+0000 7efc9ffff700 1 --2- 172.21.15.116:0/2819847668 >> v2:172.21.15.40:3300/0 conn(0x7efca00805f0 0x7efca00b5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.381+0000 7efc9ffff700 1 -- 172.21.15.116:0/2819847668 >> v1:172.21.15.40:6789/0 conn(0x7efca00b6390 legacy=0x7efca0157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:08.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.381+0000 7efc9ffff700 1 -- 172.21.15.116:0/2819847668 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca0158300 con 0x7efca00b3010 2020-11-20T13:56:08.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.381+0000 7efc9ffff700 1 --2- 172.21.15.116:0/2819847668 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efca00b3010 0x7efca00b34c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7efca0067d50 tx=0x7efc98009270).ready entity=mon.2 client_cookie=6f0a0c2c3d23c2dc server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:08.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.381+0000 7efc9e7fc700 1 -- 172.21.15.116:0/2819847668 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc9800bbc0 con 0x7efca00b3010 2020-11-20T13:56:08.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.381+0000 7efc9e7fc700 1 -- 172.21.15.116:0/2819847668 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc9800bd20 con 0x7efca00b3010 2020-11-20T13:56:08.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.381+0000 7efc9e7fc700 1 -- 172.21.15.116:0/2819847668 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc98011400 con 0x7efca00b3010 2020-11-20T13:56:08.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.381+0000 7efca533b700 1 -- 172.21.15.116:0/2819847668 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efca00b3010 msgr2=0x7efca00b34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:08.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.381+0000 7efca533b700 1 --2- 172.21.15.116:0/2819847668 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efca00b3010 0x7efca00b34c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7efca0067d50 tx=0x7efc98009270).stop 2020-11-20T13:56:08.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.382+0000 7efca533b700 1 -- 172.21.15.116:0/2819847668 shutdown_connections 2020-11-20T13:56:08.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.382+0000 7efca533b700 1 --2- 172.21.15.116:0/2819847668 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efca00b3010 0x7efca00b34c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.382+0000 7efca533b700 1 --2- 172.21.15.116:0/2819847668 >> v2:172.21.15.40:3300/0 conn(0x7efca00805f0 0x7efca00b5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.382+0000 7efca533b700 1 -- 172.21.15.116:0/2819847668 >> 172.21.15.116:0/2819847668 conn(0x7efca005f390 msgr2=0x7efca005f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:08.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.382+0000 7efca533b700 1 -- 172.21.15.116:0/2819847668 shutdown_connections 2020-11-20T13:56:08.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.382+0000 7efca533b700 1 -- 172.21.15.116:0/2819847668 wait complete. 2020-11-20T13:56:08.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.383+0000 7efca533b700 1 Processor -- start 2020-11-20T13:56:08.394 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.383+0000 7efca533b700 1 -- start start 2020-11-20T13:56:08.394 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.384+0000 7efca533b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efca00805f0 0x7efca0138650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:08.394 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.384+0000 7efca533b700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efca00b3010 0x7efca0155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:08.394 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.384+0000 7efca533b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efca0158600 0x7efca0064e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:08.395 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.384+0000 7efca533b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca0156490 con 0x7efca00805f0 2020-11-20T13:56:08.395 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.384+0000 7efca533b700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca0137c70 con 0x7efca0158600 2020-11-20T13:56:08.395 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.384+0000 7efca533b700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efca0137f10 con 0x7efca00b3010 2020-11-20T13:56:08.395 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.384+0000 7efc9effd700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efca00b3010 0x7efca0155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:08.395 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.384+0000 7efc9ffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efca0158600 0x7efca0064e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:08.396 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.384+0000 7efc9effd700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efca00b3010 0x7efca0155fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.116:40532/0 (socket says 172.21.15.116:40532) 2020-11-20T13:56:08.396 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.384+0000 7efc9effd700 1 -- 172.21.15.116:0/2976354913 learned_addr learned my addr 172.21.15.116:0/2976354913 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T13:56:08.396 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.384+0000 7efc9f7fe700 1 --2- 172.21.15.116:0/2976354913 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efca00805f0 0x7efca0138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:08.396 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.384+0000 7efc9effd700 1 -- 172.21.15.116:0/2976354913 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efca0158600 msgr2=0x7efca0064e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:08.397 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.384+0000 7efc9effd700 1 --2- 172.21.15.116:0/2976354913 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efca0158600 0x7efca0064e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.397 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.384+0000 7efc9effd700 1 -- 172.21.15.116:0/2976354913 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efca00805f0 msgr2=0x7efca0138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:08.397 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.384+0000 7efc9effd700 1 --2- 172.21.15.116:0/2976354913 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efca00805f0 0x7efca0138650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.397 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.384+0000 7efc9effd700 1 -- 172.21.15.116:0/2976354913 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca01381b0 con 0x7efca00b3010 2020-11-20T13:56:08.397 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.384+0000 7efc9f7fe700 1 --2- 172.21.15.116:0/2976354913 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efca00805f0 0x7efca0138650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:08.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.384+0000 7efc9effd700 1 --2- 172.21.15.116:0/2976354913 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efca00b3010 0x7efca0155fa0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7efc94001f50 tx=0x7efc94007b70).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:08.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.384+0000 7efc87fff700 1 -- 172.21.15.116:0/2976354913 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc9400e4a0 con 0x7efca00b3010 2020-11-20T13:56:08.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.384+0000 7efc87fff700 1 -- 172.21.15.116:0/2976354913 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc9400e600 con 0x7efca00b3010 2020-11-20T13:56:08.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.384+0000 7efc87fff700 1 -- 172.21.15.116:0/2976354913 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc9400e960 con 0x7efca00b3010 2020-11-20T13:56:08.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.384+0000 7efca533b700 1 -- 172.21.15.116:0/2976354913 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efca0065320 con 0x7efca00b3010 2020-11-20T13:56:08.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.385+0000 7efca533b700 1 -- 172.21.15.116:0/2976354913 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efca0062d90 con 0x7efca00b3010 2020-11-20T13:56:08.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.385+0000 7efc87fff700 1 -- 172.21.15.116:0/2976354913 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 2) v1 ==== 31561+0+0 (secure 0 0 0) 0x7efc94010030 con 0x7efca00b3010 2020-11-20T13:56:08.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.385+0000 7efc87fff700 1 -- 172.21.15.116:0/2976354913 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1099+0+0 (secure 0 0 0) 0x7efc94038be0 con 0x7efca00b3010 2020-11-20T13:56:08.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.386+0000 7efca533b700 1 -- 172.21.15.116:0/2976354913 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efca0057050 con 0x7efca00b3010 2020-11-20T13:56:08.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.388+0000 7efc87fff700 1 -- 172.21.15.116:0/2976354913 <== mon.2 v2:172.21.15.116:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7efc9400eac0 con 0x7efca00b3010 2020-11-20T13:56:08.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.471+0000 7efca533b700 1 -- 172.21.15.116:0/2976354913 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f4dc726b-6b34-4e8b-8eeb-87060b58002d", "id": 3} v 0) v1 -- 0x7efca00b5940 con 0x7efca00b3010 2020-11-20T13:56:08.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.475+0000 7efc87fff700 1 -- 172.21.15.116:0/2976354913 <== mon.2 v2:172.21.15.116:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f4dc726b-6b34-4e8b-8eeb-87060b58002d", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7efc9401be40 con 0x7efca00b3010 2020-11-20T13:56:08.476 INFO:teuthology.orchestra.run.smithi116.stdout:3 2020-11-20T13:56:08.478 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.476+0000 7efca533b700 1 -- 172.21.15.116:0/2976354913 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efca00b3010 msgr2=0x7efca0155fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:08.478 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.476+0000 7efca533b700 1 --2- 172.21.15.116:0/2976354913 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efca00b3010 0x7efca0155fa0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7efc94001f50 tx=0x7efc94007b70).stop 2020-11-20T13:56:08.478 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.476+0000 7efca533b700 1 -- 172.21.15.116:0/2976354913 shutdown_connections 2020-11-20T13:56:08.478 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.476+0000 7efca533b700 1 --2- 172.21.15.116:0/2976354913 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efca0158600 0x7efca0064e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.478 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.476+0000 7efca533b700 1 --2- 172.21.15.116:0/2976354913 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efca00805f0 0x7efca0138650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.476+0000 7efca533b700 1 --2- 172.21.15.116:0/2976354913 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efca00b3010 0x7efca0155fa0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.476+0000 7efca533b700 1 -- 172.21.15.116:0/2976354913 >> 172.21.15.116:0/2976354913 conn(0x7efca005f390 msgr2=0x7efca0150410 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:08.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.476+0000 7efca533b700 1 -- 172.21.15.116:0/2976354913 shutdown_connections 2020-11-20T13:56:08.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.476+0000 7efca533b700 1 -- 172.21.15.116:0/2976354913 wait complete. 2020-11-20T13:56:08.487 INFO:teuthology.orchestra.run.smithi116:> sudo ceph --cluster ceph osd new 25a6d53b-d1f0-4498-b7d5-ffa38661d403 4 2020-11-20T13:56:08.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.589+0000 7f211d646700 1 Processor -- start 2020-11-20T13:56:08.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.589+0000 7f211d646700 1 -- start start 2020-11-20T13:56:08.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.590+0000 7f211d646700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f21180805f0 0x7f21180b5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:08.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.590+0000 7f211d646700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f21180b6390 0x7f2118157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:08.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.590+0000 7f211d646700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2118157750 con 0x7f21181580e0 2020-11-20T13:56:08.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.590+0000 7f211d646700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21180809d0 con 0x7f21180805f0 2020-11-20T13:56:08.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.590+0000 7f211d646700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21180b6820 con 0x7f21180b6390 2020-11-20T13:56:08.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.590+0000 7f2116ffd700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f21180b6390 0x7f2118157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:08.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.590+0000 7f2117fff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f21181580e0 0x7f21180b3010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:50608/0 (socket says 172.21.15.116:50608) 2020-11-20T13:56:08.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.590+0000 7f2117fff700 1 -- 172.21.15.116:0/804345824 learned_addr learned my addr 172.21.15.116:0/804345824 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T13:56:08.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.590+0000 7f21177fe700 1 --2- 172.21.15.116:0/804345824 >> v2:172.21.15.40:3300/0 conn(0x7f21180805f0 0x7f21180b5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:08.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.591+0000 7f2116ffd700 1 -- 172.21.15.116:0/804345824 >> v2:172.21.15.40:3300/0 conn(0x7f21180805f0 msgr2=0x7f21180b5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:08.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.591+0000 7f2116ffd700 1 --2- 172.21.15.116:0/804345824 >> v2:172.21.15.40:3300/0 conn(0x7f21180805f0 0x7f21180b5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.591+0000 7f21167fc700 1 -- 172.21.15.116:0/804345824 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3282255092 0 0) 0x7f2118157750 con 0x7f21181580e0 2020-11-20T13:56:08.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.591+0000 7f2116ffd700 1 -- 172.21.15.116:0/804345824 >> v1:172.21.15.116:6789/0 conn(0x7f21181580e0 legacy=0x7f21180b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:08.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.591+0000 7f2116ffd700 1 -- 172.21.15.116:0/804345824 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21180b18c0 con 0x7f21180b6390 2020-11-20T13:56:08.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.591+0000 7f21177fe700 1 --2- 172.21.15.116:0/804345824 >> v2:172.21.15.40:3300/0 conn(0x7f21180805f0 0x7f21180b5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:08.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.591+0000 7f2116ffd700 1 --2- 172.21.15.116:0/804345824 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f21180b6390 0x7f2118157bf0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f210c002f30 tx=0x7f210c006dc0).ready entity=mon.2 client_cookie=e86792d3ae166c29 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:08.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.591+0000 7f21167fc700 1 -- 172.21.15.116:0/804345824 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f210c0097f0 con 0x7f21180b6390 2020-11-20T13:56:08.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.591+0000 7f21167fc700 1 -- 172.21.15.116:0/804345824 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f210c009950 con 0x7f21180b6390 2020-11-20T13:56:08.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.591+0000 7f21167fc700 1 -- 172.21.15.116:0/804345824 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f210c009cb0 con 0x7f21180b6390 2020-11-20T13:56:08.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.591+0000 7f211d646700 1 -- 172.21.15.116:0/804345824 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f21180b6390 msgr2=0x7f2118157bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:08.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.591+0000 7f211d646700 1 --2- 172.21.15.116:0/804345824 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f21180b6390 0x7f2118157bf0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f210c002f30 tx=0x7f210c006dc0).stop 2020-11-20T13:56:08.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.591+0000 7f211d646700 1 -- 172.21.15.116:0/804345824 shutdown_connections 2020-11-20T13:56:08.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.591+0000 7f211d646700 1 --2- 172.21.15.116:0/804345824 >> v2:172.21.15.40:3300/0 conn(0x7f21180805f0 0x7f21180b5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.591+0000 7f211d646700 1 --2- 172.21.15.116:0/804345824 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f21180b6390 0x7f2118157bf0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.591+0000 7f211d646700 1 -- 172.21.15.116:0/804345824 >> 172.21.15.116:0/804345824 conn(0x7f211805f390 msgr2=0x7f211805f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:08.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.591+0000 7f211d646700 1 -- 172.21.15.116:0/804345824 shutdown_connections 2020-11-20T13:56:08.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.591+0000 7f211d646700 1 -- 172.21.15.116:0/804345824 wait complete. 2020-11-20T13:56:08.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.593+0000 7f211d646700 1 Processor -- start 2020-11-20T13:56:08.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.593+0000 7f211d646700 1 -- start start 2020-11-20T13:56:08.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.593+0000 7f211d646700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f21180805f0 0x7f21181586a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:08.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.593+0000 7f211d646700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f21181580e0 0x7f21180af010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:08.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.593+0000 7f211d646700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2118153ca0 0x7f2118138dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:08.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.593+0000 7f211d646700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2118138990 con 0x7f2118153ca0 2020-11-20T13:56:08.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.593+0000 7f211d646700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21180669e0 con 0x7f21181580e0 2020-11-20T13:56:08.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.593+0000 7f211d646700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2118066c80 con 0x7f21180805f0 2020-11-20T13:56:08.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.594+0000 7f21177fe700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f21180805f0 0x7f21181586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:08.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.594+0000 7f21177fe700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f21180805f0 0x7f21181586a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.116:40540/0 (socket says 172.21.15.116:40540) 2020-11-20T13:56:08.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.594+0000 7f21177fe700 1 -- 172.21.15.116:0/2599555239 learned_addr learned my addr 172.21.15.116:0/2599555239 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T13:56:08.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.594+0000 7f2116ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f21181580e0 0x7f21180af010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:08.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.594+0000 7f2117fff700 1 --2- 172.21.15.116:0/2599555239 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2118153ca0 0x7f2118138dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:08.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.594+0000 7f21177fe700 1 -- 172.21.15.116:0/2599555239 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f21181580e0 msgr2=0x7f21180af010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:08.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.594+0000 7f21177fe700 1 --2- 172.21.15.116:0/2599555239 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f21181580e0 0x7f21180af010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.594+0000 7f21177fe700 1 -- 172.21.15.116:0/2599555239 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2118153ca0 msgr2=0x7f2118138dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:08.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.594+0000 7f21177fe700 1 --2- 172.21.15.116:0/2599555239 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2118153ca0 0x7f2118138dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.594+0000 7f21177fe700 1 -- 172.21.15.116:0/2599555239 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21180666d0 con 0x7f21180805f0 2020-11-20T13:56:08.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.594+0000 7f2116ffd700 1 --2- 172.21.15.116:0/2599555239 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f21181580e0 0x7f21180af010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:56:08.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.594+0000 7f2117fff700 1 --2- 172.21.15.116:0/2599555239 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2118153ca0 0x7f2118138dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:08.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.594+0000 7f21177fe700 1 --2- 172.21.15.116:0/2599555239 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f21180805f0 0x7f21181586a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f21181556b0 tx=0x7f2108008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:08.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.594+0000 7f20fffff700 1 -- 172.21.15.116:0/2599555239 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2108009980 con 0x7f21180805f0 2020-11-20T13:56:08.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.594+0000 7f20fffff700 1 -- 172.21.15.116:0/2599555239 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2108009ae0 con 0x7f21180805f0 2020-11-20T13:56:08.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.594+0000 7f20fffff700 1 -- 172.21.15.116:0/2599555239 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2108009e40 con 0x7f21180805f0 2020-11-20T13:56:08.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.594+0000 7f211d646700 1 -- 172.21.15.116:0/2599555239 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2118154120 con 0x7f21180805f0 2020-11-20T13:56:08.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.594+0000 7f211d646700 1 -- 172.21.15.116:0/2599555239 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f211815e440 con 0x7f21180805f0 2020-11-20T13:56:08.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.595+0000 7f20fffff700 1 -- 172.21.15.116:0/2599555239 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 2) v1 ==== 31561+0+0 (secure 0 0 0) 0x7f2108013480 con 0x7f21180805f0 2020-11-20T13:56:08.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.595+0000 7f20fffff700 1 -- 172.21.15.116:0/2599555239 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1590+0+0 (secure 0 0 0) 0x7f21080396d0 con 0x7f21180805f0 2020-11-20T13:56:08.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.595+0000 7f211d646700 1 -- 172.21.15.116:0/2599555239 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2118057050 con 0x7f21180805f0 2020-11-20T13:56:08.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.597+0000 7f20fffff700 1 -- 172.21.15.116:0/2599555239 <== mon.2 v2:172.21.15.116:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f210801a8e0 con 0x7f21180805f0 2020-11-20T13:56:08.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.679+0000 7f211d646700 1 -- 172.21.15.116:0/2599555239 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "osd new", "uuid": "25a6d53b-d1f0-4498-b7d5-ffa38661d403", "id": 4} v 0) v1 -- 0x7f211806aca0 con 0x7f21180805f0 2020-11-20T13:56:08.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.687+0000 7f20fffff700 1 -- 172.21.15.116:0/2599555239 <== mon.2 v2:172.21.15.116:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "25a6d53b-d1f0-4498-b7d5-ffa38661d403", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f210801a310 con 0x7f21180805f0 2020-11-20T13:56:08.689 INFO:teuthology.orchestra.run.smithi116.stdout:4 2020-11-20T13:56:08.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.688+0000 7f211d646700 1 -- 172.21.15.116:0/2599555239 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f21180805f0 msgr2=0x7f21181586a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:08.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.688+0000 7f211d646700 1 --2- 172.21.15.116:0/2599555239 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f21180805f0 0x7f21181586a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f21181556b0 tx=0x7f2108008040).stop 2020-11-20T13:56:08.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.688+0000 7f211d646700 1 -- 172.21.15.116:0/2599555239 shutdown_connections 2020-11-20T13:56:08.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.688+0000 7f211d646700 1 --2- 172.21.15.116:0/2599555239 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f21180805f0 0x7f21181586a0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.688+0000 7f211d646700 1 --2- 172.21.15.116:0/2599555239 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2118153ca0 0x7f2118138dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.688+0000 7f211d646700 1 --2- 172.21.15.116:0/2599555239 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f21181580e0 0x7f21180af010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.688+0000 7f211d646700 1 -- 172.21.15.116:0/2599555239 >> 172.21.15.116:0/2599555239 conn(0x7f211805f390 msgr2=0x7f21180acd60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:08.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.688+0000 7f211d646700 1 -- 172.21.15.116:0/2599555239 shutdown_connections 2020-11-20T13:56:08.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.688+0000 7f211d646700 1 -- 172.21.15.116:0/2599555239 wait complete. 2020-11-20T13:56:08.701 INFO:teuthology.orchestra.run.smithi116:> sudo ceph --cluster ceph osd new 7d7a5af8-229b-4351-9a21-a3d509cd4635 5 2020-11-20T13:56:08.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.803+0000 7f2d6ac06700 1 Processor -- start 2020-11-20T13:56:08.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.804+0000 7f2d6ac06700 1 -- start start 2020-11-20T13:56:08.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.804+0000 7f2d6ac06700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2d640805f0 0x7f2d640b5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:08.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.804+0000 7f2d6ac06700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2d640b3010 0x7f2d640b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:08.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.804+0000 7f2d6ac06700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d64157750 con 0x7f2d640b6390 2020-11-20T13:56:08.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.804+0000 7f2d6ac06700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d64157570 con 0x7f2d640b3010 2020-11-20T13:56:08.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.804+0000 7f2d6ac06700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d640809d0 con 0x7f2d640805f0 2020-11-20T13:56:08.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.804+0000 7f2d699a4700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2d640b3010 0x7f2d640b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:08.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.804+0000 7f2d691a3700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2d640805f0 0x7f2d640b5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:08.813 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.804+0000 7f2d699a4700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2d640b3010 0x7f2d640b34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48184/0 (socket says 172.21.15.116:48184) 2020-11-20T13:56:08.813 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.804+0000 7f2d691a3700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2d640805f0 0x7f2d640b5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.116:40560/0 (socket says 172.21.15.116:40560) 2020-11-20T13:56:08.813 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.804+0000 7f2d699a4700 1 -- 172.21.15.116:0/1043477266 learned_addr learned my addr 172.21.15.116:0/1043477266 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T13:56:08.813 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.805+0000 7f2d691a3700 1 -- 172.21.15.116:0/1043477266 >> v2:172.21.15.116:3300/0 conn(0x7f2d640b3010 msgr2=0x7f2d640b34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:08.814 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.805+0000 7f2d691a3700 1 --2- 172.21.15.116:0/1043477266 >> v2:172.21.15.116:3300/0 conn(0x7f2d640b3010 0x7f2d640b34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.814 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.805+0000 7f2d691a3700 1 -- 172.21.15.116:0/1043477266 >> v1:172.21.15.40:6789/0 conn(0x7f2d640b6390 legacy=0x7f2d64157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:08.814 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.805+0000 7f2d691a3700 1 -- 172.21.15.116:0/1043477266 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d64158300 con 0x7f2d640805f0 2020-11-20T13:56:08.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.805+0000 7f2d699a4700 1 --2- 172.21.15.116:0/1043477266 >> v2:172.21.15.116:3300/0 conn(0x7f2d640b3010 0x7f2d640b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:56:08.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.805+0000 7f2d691a3700 1 --2- 172.21.15.116:0/1043477266 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2d640805f0 0x7f2d640b5ea0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f2d640b3ac0 tx=0x7f2d60006dc0).ready entity=mon.2 client_cookie=928e02380831ae0f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:08.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.805+0000 7f2d5bfff700 1 -- 172.21.15.116:0/1043477266 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d60009720 con 0x7f2d640805f0 2020-11-20T13:56:08.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.805+0000 7f2d5bfff700 1 -- 172.21.15.116:0/1043477266 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d60009880 con 0x7f2d640805f0 2020-11-20T13:56:08.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.805+0000 7f2d5bfff700 1 -- 172.21.15.116:0/1043477266 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d60009be0 con 0x7f2d640805f0 2020-11-20T13:56:08.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.805+0000 7f2d6ac06700 1 -- 172.21.15.116:0/1043477266 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2d640805f0 msgr2=0x7f2d640b5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:08.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.805+0000 7f2d6ac06700 1 --2- 172.21.15.116:0/1043477266 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2d640805f0 0x7f2d640b5ea0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f2d640b3ac0 tx=0x7f2d60006dc0).stop 2020-11-20T13:56:08.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.806+0000 7f2d6ac06700 1 -- 172.21.15.116:0/1043477266 shutdown_connections 2020-11-20T13:56:08.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.806+0000 7f2d6ac06700 1 --2- 172.21.15.116:0/1043477266 >> v2:172.21.15.116:3300/0 conn(0x7f2d640b3010 0x7f2d640b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.806+0000 7f2d6ac06700 1 --2- 172.21.15.116:0/1043477266 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2d640805f0 0x7f2d640b5ea0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.806+0000 7f2d6ac06700 1 -- 172.21.15.116:0/1043477266 >> 172.21.15.116:0/1043477266 conn(0x7f2d6405f390 msgr2=0x7f2d6405f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:08.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.806+0000 7f2d6ac06700 1 -- 172.21.15.116:0/1043477266 shutdown_connections 2020-11-20T13:56:08.818 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.806+0000 7f2d6ac06700 1 -- 172.21.15.116:0/1043477266 wait complete. 2020-11-20T13:56:08.818 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.807+0000 7f2d6ac06700 1 Processor -- start 2020-11-20T13:56:08.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.807+0000 7f2d6ac06700 1 -- start start 2020-11-20T13:56:08.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.807+0000 7f2d6ac06700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2d640805f0 0x7f2d6415ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:08.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.807+0000 7f2d6ac06700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d640b3010 0x7f2d640b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:08.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.807+0000 7f2d6ac06700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d641547c0 0x7f2d640b0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:08.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.807+0000 7f2d6ac06700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d64137ad0 con 0x7f2d640b3010 2020-11-20T13:56:08.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.807+0000 7f2d6ac06700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d64137d70 con 0x7f2d641547c0 2020-11-20T13:56:08.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.807+0000 7f2d6ac06700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d64137750 con 0x7f2d640805f0 2020-11-20T13:56:08.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.808+0000 7f2d691a3700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2d640805f0 0x7f2d6415ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:08.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.808+0000 7f2d699a4700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d641547c0 0x7f2d640b0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:08.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.808+0000 7f2d691a3700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2d640805f0 0x7f2d6415ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.116:40566/0 (socket says 172.21.15.116:40566) 2020-11-20T13:56:08.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.808+0000 7f2d691a3700 1 -- 172.21.15.116:0/575336101 learned_addr learned my addr 172.21.15.116:0/575336101 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T13:56:08.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.808+0000 7f2d689a2700 1 --2- 172.21.15.116:0/575336101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d640b3010 0x7f2d640b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:08.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.808+0000 7f2d691a3700 1 -- 172.21.15.116:0/575336101 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d641547c0 msgr2=0x7f2d640b0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:08.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.808+0000 7f2d691a3700 1 --2- 172.21.15.116:0/575336101 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d641547c0 0x7f2d640b0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.808+0000 7f2d691a3700 1 -- 172.21.15.116:0/575336101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d640b3010 msgr2=0x7f2d640b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:08.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.808+0000 7f2d691a3700 1 --2- 172.21.15.116:0/575336101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d640b3010 0x7f2d640b05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.808+0000 7f2d691a3700 1 -- 172.21.15.116:0/575336101 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d641384b0 con 0x7f2d640805f0 2020-11-20T13:56:08.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.808+0000 7f2d689a2700 1 --2- 172.21.15.116:0/575336101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d640b3010 0x7f2d640b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:08.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.808+0000 7f2d691a3700 1 --2- 172.21.15.116:0/575336101 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2d640805f0 0x7f2d6415ea60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f2d640b3ac0 tx=0x7f2d60015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:08.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.808+0000 7f2d59ffb700 1 -- 172.21.15.116:0/575336101 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d60011c10 con 0x7f2d640805f0 2020-11-20T13:56:08.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.808+0000 7f2d59ffb700 1 -- 172.21.15.116:0/575336101 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d60011d70 con 0x7f2d640805f0 2020-11-20T13:56:08.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.808+0000 7f2d59ffb700 1 -- 172.21.15.116:0/575336101 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d6001d640 con 0x7f2d640805f0 2020-11-20T13:56:08.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.808+0000 7f2d6ac06700 1 -- 172.21.15.116:0/575336101 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d64067370 con 0x7f2d640805f0 2020-11-20T13:56:08.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.808+0000 7f2d6ac06700 1 -- 172.21.15.116:0/575336101 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d640b1140 con 0x7f2d640805f0 2020-11-20T13:56:08.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.809+0000 7f2d59ffb700 1 -- 172.21.15.116:0/575336101 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 2) v1 ==== 31561+0+0 (secure 0 0 0) 0x7f2d60024030 con 0x7f2d640805f0 2020-11-20T13:56:08.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.809+0000 7f2d59ffb700 1 -- 172.21.15.116:0/575336101 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1729+0+0 (secure 0 0 0) 0x7f2d60039f30 con 0x7f2d640805f0 2020-11-20T13:56:08.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.809+0000 7f2d6ac06700 1 -- 172.21.15.116:0/575336101 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d64137910 con 0x7f2d640805f0 2020-11-20T13:56:08.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.811+0000 7f2d59ffb700 1 -- 172.21.15.116:0/575336101 <== mon.2 v2:172.21.15.116:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f2d60022020 con 0x7f2d640805f0 2020-11-20T13:56:08.897 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.896+0000 7f2d6ac06700 1 -- 172.21.15.116:0/575336101 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7d7a5af8-229b-4351-9a21-a3d509cd4635", "id": 5} v 0) v1 -- 0x7f2d64057050 con 0x7f2d640805f0 2020-11-20T13:56:08.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.902+0000 7f2d59ffb700 1 -- 172.21.15.116:0/575336101 <== mon.2 v2:172.21.15.116:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7d7a5af8-229b-4351-9a21-a3d509cd4635", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2d60022020 con 0x7f2d640805f0 2020-11-20T13:56:08.904 INFO:teuthology.orchestra.run.smithi116.stdout:5 2020-11-20T13:56:08.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.903+0000 7f2d6ac06700 1 -- 172.21.15.116:0/575336101 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2d640805f0 msgr2=0x7f2d6415ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:08.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.903+0000 7f2d6ac06700 1 --2- 172.21.15.116:0/575336101 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2d640805f0 0x7f2d6415ea60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f2d640b3ac0 tx=0x7f2d60015000).stop 2020-11-20T13:56:08.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.903+0000 7f2d6ac06700 1 -- 172.21.15.116:0/575336101 shutdown_connections 2020-11-20T13:56:08.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.903+0000 7f2d6ac06700 1 --2- 172.21.15.116:0/575336101 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2d640805f0 0x7f2d6415ea60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.903+0000 7f2d6ac06700 1 --2- 172.21.15.116:0/575336101 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d641547c0 0x7f2d640b0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.903+0000 7f2d6ac06700 1 --2- 172.21.15.116:0/575336101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d640b3010 0x7f2d640b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.903+0000 7f2d6ac06700 1 -- 172.21.15.116:0/575336101 >> 172.21.15.116:0/575336101 conn(0x7f2d6405f390 msgr2=0x7f2d640b7420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:08.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.903+0000 7f2d6ac06700 1 -- 172.21.15.116:0/575336101 shutdown_connections 2020-11-20T13:56:08.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:08.903+0000 7f2d6ac06700 1 -- 172.21.15.116:0/575336101 wait complete. 2020-11-20T13:56:08.915 INFO:teuthology.orchestra.run.smithi116:> sudo ceph --cluster ceph osd new 8247d248-a140-4e36-9796-9847a97df58d 6 2020-11-20T13:56:09.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.017+0000 7fa3a3335700 1 Processor -- start 2020-11-20T13:56:09.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.017+0000 7fa3a3335700 1 -- start start 2020-11-20T13:56:09.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.018+0000 7fa3a3335700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa39c0805f0 0x7fa39c0b5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:09.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.018+0000 7fa3a3335700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa39c0b3010 0x7fa39c0b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:09.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.018+0000 7fa3a3335700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa39c157750 con 0x7fa39c0b6390 2020-11-20T13:56:09.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.018+0000 7fa3a3335700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa39c157570 con 0x7fa39c0805f0 2020-11-20T13:56:09.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.018+0000 7fa3a3335700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c0809d0 con 0x7fa39c0b3010 2020-11-20T13:56:09.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.018+0000 7fa3a18d2700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa39c0805f0 0x7fa39c0b5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:09.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.018+0000 7fa3a20d3700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa39c0b3010 0x7fa39c0b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:09.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.018+0000 7fa3a20d3700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa39c0b3010 0x7fa39c0b34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.116:40572/0 (socket says 172.21.15.116:40572) 2020-11-20T13:56:09.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.018+0000 7fa3a18d2700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa39c0805f0 0x7fa39c0b5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48192/0 (socket says 172.21.15.116:48192) 2020-11-20T13:56:09.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.018+0000 7fa3a20d3700 1 -- 172.21.15.116:0/626384222 learned_addr learned my addr 172.21.15.116:0/626384222 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T13:56:09.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.018+0000 7fa3a20d3700 1 -- 172.21.15.116:0/626384222 >> v2:172.21.15.116:3300/0 conn(0x7fa39c0805f0 msgr2=0x7fa39c0b5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:09.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.018+0000 7fa3a20d3700 1 --2- 172.21.15.116:0/626384222 >> v2:172.21.15.116:3300/0 conn(0x7fa39c0805f0 0x7fa39c0b5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:09.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.018+0000 7fa3a20d3700 1 -- 172.21.15.116:0/626384222 >> v1:172.21.15.40:6789/0 conn(0x7fa39c0b6390 legacy=0x7fa39c157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:09.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.018+0000 7fa3a20d3700 1 -- 172.21.15.116:0/626384222 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa39c158300 con 0x7fa39c0b3010 2020-11-20T13:56:09.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.018+0000 7fa3a18d2700 1 --2- 172.21.15.116:0/626384222 >> v2:172.21.15.116:3300/0 conn(0x7fa39c0805f0 0x7fa39c0b5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:56:09.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.019+0000 7fa3a20d3700 1 --2- 172.21.15.116:0/626384222 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa39c0b3010 0x7fa39c0b34c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fa3940032b0 tx=0x7fa394009270).ready entity=mon.2 client_cookie=236f4a8cb15219c2 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:09.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.019+0000 7fa3a08d0700 1 -- 172.21.15.116:0/626384222 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa394009e60 con 0x7fa39c0b3010 2020-11-20T13:56:09.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.019+0000 7fa3a08d0700 1 -- 172.21.15.116:0/626384222 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa39400bc00 con 0x7fa39c0b3010 2020-11-20T13:56:09.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.019+0000 7fa3a08d0700 1 -- 172.21.15.116:0/626384222 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa39400bd70 con 0x7fa39c0b3010 2020-11-20T13:56:09.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.019+0000 7fa3a3335700 1 -- 172.21.15.116:0/626384222 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa39c0b3010 msgr2=0x7fa39c0b34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:09.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.019+0000 7fa3a3335700 1 --2- 172.21.15.116:0/626384222 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa39c0b3010 0x7fa39c0b34c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fa3940032b0 tx=0x7fa394009270).stop 2020-11-20T13:56:09.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.019+0000 7fa3a3335700 1 -- 172.21.15.116:0/626384222 shutdown_connections 2020-11-20T13:56:09.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.019+0000 7fa3a3335700 1 --2- 172.21.15.116:0/626384222 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa39c0b3010 0x7fa39c0b34c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:09.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.019+0000 7fa3a3335700 1 --2- 172.21.15.116:0/626384222 >> v2:172.21.15.116:3300/0 conn(0x7fa39c0805f0 0x7fa39c0b5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:09.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.019+0000 7fa3a3335700 1 -- 172.21.15.116:0/626384222 >> 172.21.15.116:0/626384222 conn(0x7fa39c05f390 msgr2=0x7fa39c05f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:09.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.019+0000 7fa3a3335700 1 -- 172.21.15.116:0/626384222 shutdown_connections 2020-11-20T13:56:09.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.019+0000 7fa3a3335700 1 -- 172.21.15.116:0/626384222 wait complete. 2020-11-20T13:56:09.034 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.020+0000 7fa3a3335700 1 Processor -- start 2020-11-20T13:56:09.035 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.020+0000 7fa3a3335700 1 -- start start 2020-11-20T13:56:09.035 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.021+0000 7fa3a3335700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa39c0805f0 0x7fa39c158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:09.035 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.021+0000 7fa3a3335700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa39c0b3010 0x7fa39c0b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:09.035 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.021+0000 7fa3a3335700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa39c139930 0x7fa39c066910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:09.036 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.021+0000 7fa3a3335700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c158b80 con 0x7fa39c0b3010 2020-11-20T13:56:09.036 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.021+0000 7fa3a3335700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c153af0 con 0x7fa39c0805f0 2020-11-20T13:56:09.036 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.021+0000 7fa3a3335700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c153d90 con 0x7fa39c139930 2020-11-20T13:56:09.036 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.021+0000 7fa3a20d3700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa39c139930 0x7fa39c066910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:09.037 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.021+0000 7fa3a18d2700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa39c0805f0 0x7fa39c158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:09.037 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.021+0000 7fa3a18d2700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa39c0805f0 0x7fa39c158550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48200/0 (socket says 172.21.15.116:48200) 2020-11-20T13:56:09.037 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.021+0000 7fa3a20d3700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa39c139930 0x7fa39c066910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.116:40578/0 (socket says 172.21.15.116:40578) 2020-11-20T13:56:09.037 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.021+0000 7fa3a18d2700 1 -- 172.21.15.116:0/2896017614 learned_addr learned my addr 172.21.15.116:0/2896017614 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T13:56:09.038 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.021+0000 7fa3a10d1700 1 --2- 172.21.15.116:0/2896017614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa39c0b3010 0x7fa39c0b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:09.038 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.021+0000 7fa3a18d2700 1 -- 172.21.15.116:0/2896017614 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa39c139930 msgr2=0x7fa39c066910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:09.038 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.021+0000 7fa3a18d2700 1 --2- 172.21.15.116:0/2896017614 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa39c139930 0x7fa39c066910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:09.038 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.021+0000 7fa3a18d2700 1 -- 172.21.15.116:0/2896017614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa39c0b3010 msgr2=0x7fa39c0b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:09.038 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.021+0000 7fa3a18d2700 1 --2- 172.21.15.116:0/2896017614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa39c0b3010 0x7fa39c0b05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:09.039 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.021+0000 7fa3a18d2700 1 -- 172.21.15.116:0/2896017614 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa39c138c10 con 0x7fa39c0805f0 2020-11-20T13:56:09.039 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.021+0000 7fa3a20d3700 1 --2- 172.21.15.116:0/2896017614 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa39c139930 0x7fa39c066910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:56:09.039 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.022+0000 7fa3a18d2700 1 --2- 172.21.15.116:0/2896017614 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa39c0805f0 0x7fa39c158550 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7fa38c00c000 tx=0x7fa38c006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:09.039 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.022+0000 7fa39a7fc700 1 -- 172.21.15.116:0/2896017614 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa38c00cb70 con 0x7fa39c0805f0 2020-11-20T13:56:09.039 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.022+0000 7fa39a7fc700 1 -- 172.21.15.116:0/2896017614 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa38c00ccd0 con 0x7fa39c0805f0 2020-11-20T13:56:09.040 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.022+0000 7fa3a3335700 1 -- 172.21.15.116:0/2896017614 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa39c0b0a90 con 0x7fa39c0805f0 2020-11-20T13:56:09.040 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.022+0000 7fa39a7fc700 1 -- 172.21.15.116:0/2896017614 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa38c0125e0 con 0x7fa39c0805f0 2020-11-20T13:56:09.041 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.022+0000 7fa3a3335700 1 -- 172.21.15.116:0/2896017614 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa39c0b0d90 con 0x7fa39c0805f0 2020-11-20T13:56:09.041 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.022+0000 7fa39a7fc700 1 -- 172.21.15.116:0/2896017614 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 2) v1 ==== 31561+0+0 (secure 0 0 0) 0x7fa38c012740 con 0x7fa39c0805f0 2020-11-20T13:56:09.042 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.022+0000 7fa39a7fc700 1 -- 172.21.15.116:0/2896017614 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1844+0+0 (secure 0 0 0) 0x7fa38c0396c0 con 0x7fa39c0805f0 2020-11-20T13:56:09.043 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.023+0000 7fa3a3335700 1 -- 172.21.15.116:0/2896017614 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa39c057050 con 0x7fa39c0805f0 2020-11-20T13:56:09.043 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.025+0000 7fa39a7fc700 1 -- 172.21.15.116:0/2896017614 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7fa38c0129d0 con 0x7fa39c0805f0 2020-11-20T13:56:09.109 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.108+0000 7fa3a3335700 1 -- 172.21.15.116:0/2896017614 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8247d248-a140-4e36-9796-9847a97df58d", "id": 6} v 0) v1 -- 0x7fa39c0b5940 con 0x7fa39c0805f0 2020-11-20T13:56:09.117 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.116+0000 7fa39a7fc700 1 -- 172.21.15.116:0/2896017614 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8247d248-a140-4e36-9796-9847a97df58d", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa38c0102b0 con 0x7fa39c0805f0 2020-11-20T13:56:09.118 INFO:teuthology.orchestra.run.smithi116.stdout:6 2020-11-20T13:56:09.119 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.117+0000 7fa3a3335700 1 -- 172.21.15.116:0/2896017614 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa39c0805f0 msgr2=0x7fa39c158550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:09.119 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.117+0000 7fa3a3335700 1 --2- 172.21.15.116:0/2896017614 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa39c0805f0 0x7fa39c158550 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7fa38c00c000 tx=0x7fa38c006dc0).stop 2020-11-20T13:56:09.119 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.117+0000 7fa3a3335700 1 -- 172.21.15.116:0/2896017614 shutdown_connections 2020-11-20T13:56:09.120 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.117+0000 7fa3a3335700 1 --2- 172.21.15.116:0/2896017614 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa39c139930 0x7fa39c066910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:09.120 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.117+0000 7fa3a3335700 1 --2- 172.21.15.116:0/2896017614 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa39c0805f0 0x7fa39c158550 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:09.120 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.117+0000 7fa3a3335700 1 --2- 172.21.15.116:0/2896017614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa39c0b3010 0x7fa39c0b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:09.120 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.117+0000 7fa3a3335700 1 -- 172.21.15.116:0/2896017614 >> 172.21.15.116:0/2896017614 conn(0x7fa39c05f390 msgr2=0x7fa39c0ace80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:09.120 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.117+0000 7fa3a3335700 1 -- 172.21.15.116:0/2896017614 shutdown_connections 2020-11-20T13:56:09.121 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.117+0000 7fa3a3335700 1 -- 172.21.15.116:0/2896017614 wait complete. 2020-11-20T13:56:09.129 INFO:teuthology.orchestra.run.smithi116:> sudo ceph --cluster ceph osd new 472803c4-70b8-499d-b18a-71aaeec6aab8 7 2020-11-20T13:56:09.235 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.231+0000 7f82ba65b700 1 Processor -- start 2020-11-20T13:56:09.236 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.231+0000 7f82ba65b700 1 -- start start 2020-11-20T13:56:09.236 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.232+0000 7f82ba65b700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f82b40805f0 0x7f82b40b5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:09.236 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.232+0000 7f82ba65b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f82b40b3010 0x7f82b40b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:09.236 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.232+0000 7f82ba65b700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82b4157750 con 0x7f82b40b6390 2020-11-20T13:56:09.237 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.232+0000 7f82ba65b700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82b4157570 con 0x7f82b40b3010 2020-11-20T13:56:09.237 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.232+0000 7f82ba65b700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82b40809d0 con 0x7f82b40805f0 2020-11-20T13:56:09.237 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.232+0000 7f82b8bf8700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f82b40805f0 0x7f82b40b5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:09.237 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.232+0000 7f82b8bf8700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f82b40805f0 0x7f82b40b5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.116:40582/0 (socket says 172.21.15.116:40582) 2020-11-20T13:56:09.238 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.232+0000 7f82b8bf8700 1 -- 172.21.15.116:0/3084297411 learned_addr learned my addr 172.21.15.116:0/3084297411 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T13:56:09.238 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.232+0000 7f82b93f9700 1 --2- 172.21.15.116:0/3084297411 >> v2:172.21.15.40:3300/0 conn(0x7f82b40b3010 0x7f82b40b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:09.238 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.232+0000 7f82b8bf8700 1 -- 172.21.15.116:0/3084297411 >> v2:172.21.15.40:3300/0 conn(0x7f82b40b3010 msgr2=0x7f82b40b34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:09.238 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.232+0000 7f82b8bf8700 1 --2- 172.21.15.116:0/3084297411 >> v2:172.21.15.40:3300/0 conn(0x7f82b40b3010 0x7f82b40b34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:09.238 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.232+0000 7f82b8bf8700 1 -- 172.21.15.116:0/3084297411 >> v1:172.21.15.40:6789/0 conn(0x7f82b40b6390 legacy=0x7f82b4157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:09.239 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.232+0000 7f82b8bf8700 1 -- 172.21.15.116:0/3084297411 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b4158300 con 0x7f82b40805f0 2020-11-20T13:56:09.239 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.233+0000 7f82b8bf8700 1 --2- 172.21.15.116:0/3084297411 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f82b40805f0 0x7f82b40b5ea0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f82b40b3ac0 tx=0x7f82a4006e80).ready entity=mon.2 client_cookie=8cbe8375ac597c60 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:09.239 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.233+0000 7f82b37fe700 1 -- 172.21.15.116:0/3084297411 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82a40097b0 con 0x7f82b40805f0 2020-11-20T13:56:09.239 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.233+0000 7f82b37fe700 1 -- 172.21.15.116:0/3084297411 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a4009910 con 0x7f82b40805f0 2020-11-20T13:56:09.240 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.233+0000 7f82b37fe700 1 -- 172.21.15.116:0/3084297411 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82a4009c70 con 0x7f82b40805f0 2020-11-20T13:56:09.240 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.233+0000 7f82ba65b700 1 -- 172.21.15.116:0/3084297411 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f82b40805f0 msgr2=0x7f82b40b5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:09.240 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.233+0000 7f82ba65b700 1 --2- 172.21.15.116:0/3084297411 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f82b40805f0 0x7f82b40b5ea0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f82b40b3ac0 tx=0x7f82a4006e80).stop 2020-11-20T13:56:09.240 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.233+0000 7f82ba65b700 1 -- 172.21.15.116:0/3084297411 shutdown_connections 2020-11-20T13:56:09.240 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.233+0000 7f82ba65b700 1 --2- 172.21.15.116:0/3084297411 >> v2:172.21.15.40:3300/0 conn(0x7f82b40b3010 0x7f82b40b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:09.241 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.233+0000 7f82ba65b700 1 --2- 172.21.15.116:0/3084297411 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f82b40805f0 0x7f82b40b5ea0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:09.241 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.233+0000 7f82ba65b700 1 -- 172.21.15.116:0/3084297411 >> 172.21.15.116:0/3084297411 conn(0x7f82b405f390 msgr2=0x7f82b405f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:09.241 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.233+0000 7f82ba65b700 1 -- 172.21.15.116:0/3084297411 shutdown_connections 2020-11-20T13:56:09.241 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.233+0000 7f82ba65b700 1 -- 172.21.15.116:0/3084297411 wait complete. 2020-11-20T13:56:09.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.235+0000 7f82ba65b700 1 Processor -- start 2020-11-20T13:56:09.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.235+0000 7f82ba65b700 1 -- start start 2020-11-20T13:56:09.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.235+0000 7f82ba65b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82b40805f0 0x7f82b413b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:09.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.235+0000 7f82ba65b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82b40b3010 0x7f82b413b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:09.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.235+0000 7f82ba65b700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f82b4067260 0x7f82b4064030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:09.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.235+0000 7f82ba65b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b414e6a0 con 0x7f82b40b3010 2020-11-20T13:56:09.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.235+0000 7f82ba65b700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b4066dc0 con 0x7f82b40805f0 2020-11-20T13:56:09.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.235+0000 7f82ba65b700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82b413c0c0 con 0x7f82b4067260 2020-11-20T13:56:09.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.235+0000 7f82b8bf8700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82b40805f0 0x7f82b413b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:09.246 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.235+0000 7f82b93f9700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f82b4067260 0x7f82b4064030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:09.246 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.235+0000 7f82b8bf8700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82b40805f0 0x7f82b413b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48206/0 (socket says 172.21.15.116:48206) 2020-11-20T13:56:09.246 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.236+0000 7f82b8bf8700 1 -- 172.21.15.116:0/4212376225 learned_addr learned my addr 172.21.15.116:0/4212376225 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T13:56:09.246 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.235+0000 7f82b93f9700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f82b4067260 0x7f82b4064030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.116:40590/0 (socket says 172.21.15.116:40590) 2020-11-20T13:56:09.246 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.236+0000 7f82b3fff700 1 --2- 172.21.15.116:0/4212376225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82b40b3010 0x7f82b413b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:09.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.236+0000 7f82b8bf8700 1 -- 172.21.15.116:0/4212376225 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f82b4067260 msgr2=0x7f82b4064030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:09.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.236+0000 7f82b8bf8700 1 --2- 172.21.15.116:0/4212376225 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f82b4067260 0x7f82b4064030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:09.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.236+0000 7f82b8bf8700 1 -- 172.21.15.116:0/4212376225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82b40b3010 msgr2=0x7f82b413b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:09.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.236+0000 7f82b8bf8700 1 --2- 172.21.15.116:0/4212376225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82b40b3010 0x7f82b413b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:09.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.236+0000 7f82b8bf8700 1 -- 172.21.15.116:0/4212376225 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b414e990 con 0x7f82b40805f0 2020-11-20T13:56:09.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.236+0000 7f82b93f9700 1 --2- 172.21.15.116:0/4212376225 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f82b4067260 0x7f82b4064030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:56:09.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.236+0000 7f82b8bf8700 1 --2- 172.21.15.116:0/4212376225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82b40805f0 0x7f82b413b4a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f82b40b3ac0 tx=0x7f82a4015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:09.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.236+0000 7f82b17fa700 1 -- 172.21.15.116:0/4212376225 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82a4011b80 con 0x7f82b40805f0 2020-11-20T13:56:09.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.236+0000 7f82b17fa700 1 -- 172.21.15.116:0/4212376225 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a4011ce0 con 0x7f82b40805f0 2020-11-20T13:56:09.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.236+0000 7f82b17fa700 1 -- 172.21.15.116:0/4212376225 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82a401e650 con 0x7f82b40805f0 2020-11-20T13:56:09.249 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.236+0000 7f82ba65b700 1 -- 172.21.15.116:0/4212376225 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82b414eb20 con 0x7f82b40805f0 2020-11-20T13:56:09.249 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.236+0000 7f82ba65b700 1 -- 172.21.15.116:0/4212376225 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82b414f6f0 con 0x7f82b40805f0 2020-11-20T13:56:09.249 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.237+0000 7f82b17fa700 1 -- 172.21.15.116:0/4212376225 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 2) v1 ==== 31561+0+0 (secure 0 0 0) 0x7f82a4025030 con 0x7f82b40805f0 2020-11-20T13:56:09.249 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.237+0000 7f82b17fa700 1 -- 172.21.15.116:0/4212376225 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1959+0+0 (secure 0 0 0) 0x7f82a403a310 con 0x7f82b40805f0 2020-11-20T13:56:09.249 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.237+0000 7f82ba65b700 1 -- 172.21.15.116:0/4212376225 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82b4057050 con 0x7f82b40805f0 2020-11-20T13:56:09.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.239+0000 7f82b17fa700 1 -- 172.21.15.116:0/4212376225 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f82a4019bc0 con 0x7f82b40805f0 2020-11-20T13:56:09.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.294+0000 7f82b17fa700 1 -- 172.21.15.116:0/4212376225 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mgrmap(e 3) v1 ==== 63226+0+0 (secure 0 0 0) 0x7f82a401a040 con 0x7f82b40805f0 2020-11-20T13:56:09.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.295+0000 7f82b17fa700 1 --2- 172.21.15.116:0/4212376225 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f8294051960 0x7f8294053d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:09.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.295+0000 7f82b3fff700 1 --2- 172.21.15.116:0/4212376225 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f8294051960 0x7f8294053d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:09.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.295+0000 7f82b3fff700 1 --2- 172.21.15.116:0/4212376225 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f8294051960 0x7f8294053d70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f82a800d000 tx=0x7f82a8000b80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:09.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.327+0000 7f82ba65b700 1 -- 172.21.15.116:0/4212376225 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd new", "uuid": "472803c4-70b8-499d-b18a-71aaeec6aab8", "id": 7} v 0) v1 -- 0x7f82b406aca0 con 0x7f82b40805f0 2020-11-20T13:56:09.332 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.331+0000 7f82b17fa700 1 -- 172.21.15.116:0/4212376225 <== mon.1 v2:172.21.15.116:3300/0 8 ==== mon_command_ack([{"prefix": "osd new", "uuid": "472803c4-70b8-499d-b18a-71aaeec6aab8", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f82a4034ef0 con 0x7f82b40805f0 2020-11-20T13:56:09.332 INFO:teuthology.orchestra.run.smithi116.stdout:7 2020-11-20T13:56:09.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.332+0000 7f82ba65b700 1 -- 172.21.15.116:0/4212376225 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f8294051960 msgr2=0x7f8294053d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:09.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.332+0000 7f82ba65b700 1 --2- 172.21.15.116:0/4212376225 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f8294051960 0x7f8294053d70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f82a800d000 tx=0x7f82a8000b80).stop 2020-11-20T13:56:09.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.332+0000 7f82ba65b700 1 -- 172.21.15.116:0/4212376225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82b40805f0 msgr2=0x7f82b413b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:09.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.332+0000 7f82ba65b700 1 --2- 172.21.15.116:0/4212376225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82b40805f0 0x7f82b413b4a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f82b40b3ac0 tx=0x7f82a4015000).stop 2020-11-20T13:56:09.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.332+0000 7f82ba65b700 1 -- 172.21.15.116:0/4212376225 shutdown_connections 2020-11-20T13:56:09.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.332+0000 7f82ba65b700 1 --2- 172.21.15.116:0/4212376225 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f8294051960 0x7f8294053d70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:09.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.332+0000 7f82ba65b700 1 --2- 172.21.15.116:0/4212376225 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f82b4067260 0x7f82b4064030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:09.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.332+0000 7f82ba65b700 1 --2- 172.21.15.116:0/4212376225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82b40805f0 0x7f82b413b4a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:09.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.332+0000 7f82ba65b700 1 --2- 172.21.15.116:0/4212376225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82b40b3010 0x7f82b413b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:09.336 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.332+0000 7f82ba65b700 1 -- 172.21.15.116:0/4212376225 >> 172.21.15.116:0/4212376225 conn(0x7f82b405f390 msgr2=0x7f82b40b7420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:09.336 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.332+0000 7f82ba65b700 1 -- 172.21.15.116:0/4212376225 shutdown_connections 2020-11-20T13:56:09.336 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T13:56:09.332+0000 7f82ba65b700 1 -- 172.21.15.116:0/4212376225 wait complete. 2020-11-20T13:56:09.343 INFO:tasks.ceph.osd.0:Restarting daemon 2020-11-20T13:56:09.344 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2020-11-20T13:56:09.346 INFO:tasks.ceph.osd.0:Started 2020-11-20T13:56:09.346 INFO:tasks.ceph.osd.1:Restarting daemon 2020-11-20T13:56:09.347 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2020-11-20T13:56:09.348 INFO:tasks.ceph.osd.1:Started 2020-11-20T13:56:09.349 INFO:tasks.ceph.osd.2:Restarting daemon 2020-11-20T13:56:09.349 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2020-11-20T13:56:09.352 INFO:tasks.ceph.osd.2:Started 2020-11-20T13:56:09.352 INFO:tasks.ceph.osd.3:Restarting daemon 2020-11-20T13:56:09.352 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2020-11-20T13:56:09.354 INFO:tasks.ceph.osd.3:Started 2020-11-20T13:56:09.355 INFO:tasks.ceph.osd.4:Restarting daemon 2020-11-20T13:56:09.355 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2020-11-20T13:56:09.357 INFO:tasks.ceph.osd.4:Started 2020-11-20T13:56:09.358 INFO:tasks.ceph.osd.5:Restarting daemon 2020-11-20T13:56:09.358 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2020-11-20T13:56:09.360 INFO:tasks.ceph.osd.5:Started 2020-11-20T13:56:09.361 INFO:tasks.ceph.osd.6:Restarting daemon 2020-11-20T13:56:09.361 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2020-11-20T13:56:09.363 INFO:tasks.ceph.osd.6:Started 2020-11-20T13:56:09.364 INFO:tasks.ceph.osd.7:Restarting daemon 2020-11-20T13:56:09.364 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2020-11-20T13:56:09.366 INFO:tasks.ceph.osd.7:Started 2020-11-20T13:56:09.366 INFO:tasks.ceph:Waiting for OSDs to come up 2020-11-20T13:56:09.953 INFO:tasks.ceph.osd.5.smithi116.stderr:2020-11-20T13:56:09.952+0000 7fc8bc74eec0 -1 Falling back to public interface 2020-11-20T13:56:09.959 INFO:tasks.ceph.osd.1.smithi040.stderr:2020-11-20T13:56:09.958+0000 7f2713b56ec0 -1 Falling back to public interface 2020-11-20T13:56:09.962 INFO:tasks.ceph.osd.6.smithi116.stderr:2020-11-20T13:56:09.961+0000 7f0bb41d7ec0 -1 Falling back to public interface 2020-11-20T13:56:09.962 INFO:tasks.ceph.osd.2.smithi040.stderr:2020-11-20T13:56:09.962+0000 7fdd41be4ec0 -1 Falling back to public interface 2020-11-20T13:56:09.968 INFO:tasks.ceph.osd.0.smithi040.stderr:2020-11-20T13:56:09.968+0000 7fc34c328ec0 -1 Falling back to public interface 2020-11-20T13:56:09.969 INFO:tasks.ceph.osd.3.smithi040.stderr:2020-11-20T13:56:09.968+0000 7f4094013ec0 -1 Falling back to public interface 2020-11-20T13:56:09.974 INFO:tasks.ceph.osd.4.smithi116.stderr:2020-11-20T13:56:09.973+0000 7fc89444bec0 -1 Falling back to public interface 2020-11-20T13:56:09.979 INFO:tasks.ceph.osd.7.smithi116.stderr:2020-11-20T13:56:09.978+0000 7f6926935ec0 -1 Falling back to public interface 2020-11-20T13:56:10.172 INFO:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2020-11-20T13:56:10.307 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.302+0000 7f2b9bf6f700 1 Processor -- start 2020-11-20T13:56:10.307 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.302+0000 7f2b9bf6f700 1 -- start start 2020-11-20T13:56:10.308 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.303+0000 7f2b9bf6f700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2b940ba0f0 0x7f2b94154720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:10.308 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.303+0000 7f2b9bf6f700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2b940800b0 0x7f2b940b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:10.308 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.303+0000 7f2b9bf6f700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b94080630 con 0x7f2b940b46f0 2020-11-20T13:56:10.308 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.303+0000 7f2b9bf6f700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b940808d0 con 0x7f2b940ba0f0 2020-11-20T13:56:10.308 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.303+0000 7f2b9bf6f700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b940ba4d0 con 0x7f2b940800b0 2020-11-20T13:56:10.309 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.303+0000 7f2b99d0b700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2b940b46f0 0x7f2b9414da10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44662/0 (socket says 172.21.15.40:44662) 2020-11-20T13:56:10.309 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.303+0000 7f2b99d0b700 1 -- 172.21.15.40:0/2498269566 learned_addr learned my addr 172.21.15.40:0/2498269566 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:10.309 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.303+0000 7f2b9ad0d700 1 --2- 172.21.15.40:0/2498269566 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2b940800b0 0x7f2b940b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:10.309 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.303+0000 7f2b9a50c700 1 --2- 172.21.15.40:0/2498269566 >> v2:172.21.15.116:3300/0 conn(0x7f2b940ba0f0 0x7f2b94154720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:10.309 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.304+0000 7f2b9950a700 1 -- 172.21.15.40:0/2498269566 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4246608088 0 0) 0x7f2b94080630 con 0x7f2b940b46f0 2020-11-20T13:56:10.310 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.304+0000 7f2b9950a700 1 -- 172.21.15.40:0/2498269566 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b80003230 con 0x7f2b940b46f0 2020-11-20T13:56:10.310 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.304+0000 7f2b9ad0d700 1 -- 172.21.15.40:0/2498269566 >> v2:172.21.15.116:3300/0 conn(0x7f2b940ba0f0 msgr2=0x7f2b94154720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:10.310 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.304+0000 7f2b9ad0d700 1 --2- 172.21.15.40:0/2498269566 >> v2:172.21.15.116:3300/0 conn(0x7f2b940ba0f0 0x7f2b94154720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:10.310 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.304+0000 7f2b9ad0d700 1 -- 172.21.15.40:0/2498269566 >> v1:172.21.15.40:6789/0 conn(0x7f2b940b46f0 legacy=0x7f2b9414da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:10.310 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.304+0000 7f2b9ad0d700 1 -- 172.21.15.40:0/2498269566 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b94080b70 con 0x7f2b940800b0 2020-11-20T13:56:10.311 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.304+0000 7f2b9a50c700 1 --2- 172.21.15.40:0/2498269566 >> v2:172.21.15.116:3300/0 conn(0x7f2b940ba0f0 0x7f2b94154720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:56:10.311 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.304+0000 7f2b9ad0d700 1 --2- 172.21.15.40:0/2498269566 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2b940800b0 0x7f2b940b60d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f2b8c0032b0 tx=0x7f2b8c0091a0).ready entity=mon.2 client_cookie=4ba6bb63301727d0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:10.311 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.304+0000 7f2b9950a700 1 -- 172.21.15.40:0/2498269566 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b8c00b940 con 0x7f2b940800b0 2020-11-20T13:56:10.311 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.304+0000 7f2b9950a700 1 -- 172.21.15.40:0/2498269566 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b8c00baa0 con 0x7f2b940800b0 2020-11-20T13:56:10.311 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.304+0000 7f2b9950a700 1 -- 172.21.15.40:0/2498269566 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b8c00bc10 con 0x7f2b940800b0 2020-11-20T13:56:10.312 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.304+0000 7f2b9bf6f700 1 -- 172.21.15.40:0/2498269566 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2b940800b0 msgr2=0x7f2b940b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:10.312 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.304+0000 7f2b9bf6f700 1 --2- 172.21.15.40:0/2498269566 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2b940800b0 0x7f2b940b60d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f2b8c0032b0 tx=0x7f2b8c0091a0).stop 2020-11-20T13:56:10.312 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.305+0000 7f2b9bf6f700 1 -- 172.21.15.40:0/2498269566 shutdown_connections 2020-11-20T13:56:10.312 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.305+0000 7f2b9bf6f700 1 --2- 172.21.15.40:0/2498269566 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2b940800b0 0x7f2b940b60d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:10.312 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.305+0000 7f2b9bf6f700 1 --2- 172.21.15.40:0/2498269566 >> v2:172.21.15.116:3300/0 conn(0x7f2b940ba0f0 0x7f2b94154720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:10.313 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.305+0000 7f2b9bf6f700 1 -- 172.21.15.40:0/2498269566 >> 172.21.15.40:0/2498269566 conn(0x7f2b9405cc60 msgr2=0x7f2b940b9660 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:10.313 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.305+0000 7f2b9bf6f700 1 -- 172.21.15.40:0/2498269566 shutdown_connections 2020-11-20T13:56:10.313 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.305+0000 7f2b9bf6f700 1 -- 172.21.15.40:0/2498269566 wait complete. 2020-11-20T13:56:10.317 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.307+0000 7f2b9bf6f700 1 Processor -- start 2020-11-20T13:56:10.317 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.307+0000 7f2b9bf6f700 1 -- start start 2020-11-20T13:56:10.317 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.308+0000 7f2b9bf6f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b940800b0 0x7f2b9413a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:10.318 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.308+0000 7f2b9bf6f700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2b940b46f0 0x7f2b940aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:10.318 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.308+0000 7f2b9bf6f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b940ba0f0 0x7f2b9415d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:10.318 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.308+0000 7f2b9bf6f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9413a8b0 con 0x7f2b940800b0 2020-11-20T13:56:10.319 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.308+0000 7f2b9bf6f700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b940ae180 con 0x7f2b940ba0f0 2020-11-20T13:56:10.319 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.308+0000 7f2b9bf6f700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b940ae420 con 0x7f2b940b46f0 2020-11-20T13:56:10.319 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.308+0000 7f2b9a50c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b940800b0 0x7f2b9413a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:10.319 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.308+0000 7f2b9a50c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b940800b0 0x7f2b9413a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54072/0 (socket says 172.21.15.40:54072) 2020-11-20T13:56:10.320 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.308+0000 7f2b9a50c700 1 -- 172.21.15.40:0/3847050767 learned_addr learned my addr 172.21.15.40:0/3847050767 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:10.320 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.308+0000 7f2b99d0b700 1 --2- 172.21.15.40:0/3847050767 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2b940b46f0 0x7f2b940aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:10.321 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.308+0000 7f2b9ad0d700 1 --2- 172.21.15.40:0/3847050767 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b940ba0f0 0x7f2b9415d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:10.321 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.308+0000 7f2b9a50c700 1 -- 172.21.15.40:0/3847050767 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2b940b46f0 msgr2=0x7f2b940aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:10.321 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.308+0000 7f2b9a50c700 1 --2- 172.21.15.40:0/3847050767 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2b940b46f0 0x7f2b940aeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:10.322 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.308+0000 7f2b9a50c700 1 -- 172.21.15.40:0/3847050767 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b940ba0f0 msgr2=0x7f2b9415d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:10.322 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.308+0000 7f2b9a50c700 1 --2- 172.21.15.40:0/3847050767 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b940ba0f0 0x7f2b9415d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:10.322 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.308+0000 7f2b9a50c700 1 -- 172.21.15.40:0/3847050767 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b940ae6c0 con 0x7f2b940800b0 2020-11-20T13:56:10.322 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.309+0000 7f2b99d0b700 1 --2- 172.21.15.40:0/3847050767 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2b940b46f0 0x7f2b940aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:56:10.322 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.309+0000 7f2b9a50c700 1 --2- 172.21.15.40:0/3847050767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b940800b0 0x7f2b9413a3c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f2b9000c000 tx=0x7f2b90006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:10.323 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.309+0000 7f2b8affd700 1 -- 172.21.15.40:0/3847050767 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b9000cae0 con 0x7f2b940800b0 2020-11-20T13:56:10.323 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.309+0000 7f2b8affd700 1 -- 172.21.15.40:0/3847050767 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b9000cc40 con 0x7f2b940800b0 2020-11-20T13:56:10.323 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.309+0000 7f2b9bf6f700 1 -- 172.21.15.40:0/3847050767 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b940af050 con 0x7f2b940800b0 2020-11-20T13:56:10.323 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.309+0000 7f2b9bf6f700 1 -- 172.21.15.40:0/3847050767 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b94156050 con 0x7f2b940800b0 2020-11-20T13:56:10.323 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.309+0000 7f2b8affd700 1 -- 172.21.15.40:0/3847050767 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b900123f0 con 0x7f2b940800b0 2020-11-20T13:56:10.324 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.311+0000 7f2b8affd700 1 -- 172.21.15.40:0/3847050767 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 3) v1 ==== 63226+0+0 (secure 0 0 0) 0x7f2b90012550 con 0x7f2b940800b0 2020-11-20T13:56:10.324 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.311+0000 7f2b8affd700 1 --2- 172.21.15.40:0/3847050767 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f2b7c051250 0x7f2b7c053710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:10.324 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.311+0000 7f2b8affd700 1 -- 172.21.15.40:0/3847050767 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 2074+0+0 (secure 0 0 0) 0x7f2b900686d0 con 0x7f2b940800b0 2020-11-20T13:56:10.324 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.311+0000 7f2b99d0b700 1 --2- 172.21.15.40:0/3847050767 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f2b7c051250 0x7f2b7c053710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:10.324 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.312+0000 7f2b9bf6f700 1 -- 172.21.15.40:0/3847050767 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b94057050 con 0x7f2b940800b0 2020-11-20T13:56:10.325 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.312+0000 7f2b99d0b700 1 --2- 172.21.15.40:0/3847050767 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f2b7c051250 0x7f2b7c053710 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f2b8400d000 tx=0x7f2b84009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:10.325 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.319+0000 7f2b8affd700 1 -- 172.21.15.40:0/3847050767 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f2b90041960 con 0x7f2b940800b0 2020-11-20T13:56:10.502 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.501+0000 7f2b9bf6f700 1 -- 172.21.15.40:0/3847050767 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b940b9640 con 0x7f2b940800b0 2020-11-20T13:56:10.503 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.501+0000 7f2b8affd700 1 -- 172.21.15.40:0/3847050767 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+8274 (secure 0 0 0) 0x7f2b90044f80 con 0x7f2b940800b0 2020-11-20T13:56:10.503 INFO:teuthology.misc.health.smithi040.stdout: 2020-11-20T13:56:10.503 INFO:teuthology.misc.health.smithi040.stdout:{"epoch":10,"fsid":"da1ed737-5174-430f-a3c9-56f6bbfad13c","created":"2020-11-20T13:56:04.990651+0000","modified":"2020-11-20T13:56:09.329616+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T13:56:08.330017+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"7","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"9ea4c667-8938-46d6-a5ca-092d55c1f79e","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"515708d9-5087-4e55-8d14-cdea1c00eacc","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4d86bca7-743e-4983-b35f-36676c405b36","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f4dc726b-6b34-4e8b-8eeb-87060b58002d","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"25a6d53b-d1f0-4498-b7d5-ffa38661d403","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7d7a5af8-229b-4351-9a21-a3d509cd4635","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"8247d248-a140-4e36-9796-9847a97df58d","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"472803c4-70b8-499d-b18a-71aaeec6aab8","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-20T13:56:10.504 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.503+0000 7f2b9bf6f700 1 -- 172.21.15.40:0/3847050767 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f2b7c051250 msgr2=0x7f2b7c053710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:10.505 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.503+0000 7f2b9bf6f700 1 --2- 172.21.15.40:0/3847050767 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f2b7c051250 0x7f2b7c053710 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f2b8400d000 tx=0x7f2b84009040).stop 2020-11-20T13:56:10.505 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.503+0000 7f2b9bf6f700 1 -- 172.21.15.40:0/3847050767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b940800b0 msgr2=0x7f2b9413a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:10.505 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.503+0000 7f2b9bf6f700 1 --2- 172.21.15.40:0/3847050767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b940800b0 0x7f2b9413a3c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f2b9000c000 tx=0x7f2b90006dc0).stop 2020-11-20T13:56:10.505 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.503+0000 7f2b9bf6f700 1 -- 172.21.15.40:0/3847050767 shutdown_connections 2020-11-20T13:56:10.506 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.503+0000 7f2b9bf6f700 1 --2- 172.21.15.40:0/3847050767 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f2b7c051250 0x7f2b7c053710 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:10.506 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.503+0000 7f2b9bf6f700 1 --2- 172.21.15.40:0/3847050767 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b940ba0f0 0x7f2b9415d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:10.506 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.503+0000 7f2b9bf6f700 1 --2- 172.21.15.40:0/3847050767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b940800b0 0x7f2b9413a3c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:10.506 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.503+0000 7f2b9bf6f700 1 --2- 172.21.15.40:0/3847050767 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2b940b46f0 0x7f2b940aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:10.506 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.503+0000 7f2b9bf6f700 1 -- 172.21.15.40:0/3847050767 >> 172.21.15.40:0/3847050767 conn(0x7f2b9405cc60 msgr2=0x7f2b940b7560 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:10.507 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.503+0000 7f2b9bf6f700 1 -- 172.21.15.40:0/3847050767 shutdown_connections 2020-11-20T13:56:10.507 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:10.503+0000 7f2b9bf6f700 1 -- 172.21.15.40:0/3847050767 wait complete. 2020-11-20T13:56:10.515 DEBUG:teuthology.misc:0 of 8 OSDs are up 2020-11-20T13:56:11.029 INFO:tasks.ceph.osd.5.smithi116.stderr:2020-11-20T13:56:11.026+0000 7fc8bc74eec0 -1 osd.5 0 log_to_monitors {default=true} 2020-11-20T13:56:11.036 INFO:tasks.ceph.osd.1.smithi040.stderr:2020-11-20T13:56:11.035+0000 7f2713b56ec0 -1 osd.1 0 log_to_monitors {default=true} 2020-11-20T13:56:11.047 INFO:tasks.ceph.osd.6.smithi116.stderr:2020-11-20T13:56:11.045+0000 7f0bb41d7ec0 -1 osd.6 0 log_to_monitors {default=true} 2020-11-20T13:56:11.048 INFO:tasks.ceph.osd.2.smithi040.stderr:2020-11-20T13:56:11.047+0000 7fdd41be4ec0 -1 osd.2 0 log_to_monitors {default=true} 2020-11-20T13:56:11.050 INFO:tasks.ceph.osd.3.smithi040.stderr:2020-11-20T13:56:11.049+0000 7f4094013ec0 -1 osd.3 0 log_to_monitors {default=true} 2020-11-20T13:56:11.054 INFO:tasks.ceph.osd.0.smithi040.stderr:2020-11-20T13:56:11.053+0000 7fc34c328ec0 -1 osd.0 0 log_to_monitors {default=true} 2020-11-20T13:56:11.058 INFO:tasks.ceph.osd.7.smithi116.stderr:2020-11-20T13:56:11.057+0000 7f6926935ec0 -1 osd.7 0 log_to_monitors {default=true} 2020-11-20T13:56:11.059 INFO:tasks.ceph.osd.4.smithi116.stderr:2020-11-20T13:56:11.057+0000 7fc89444bec0 -1 osd.4 0 log_to_monitors {default=true} 2020-11-20T13:56:12.892 INFO:tasks.ceph.osd.0.smithi040.stderr:2020-11-20T13:56:12.892+0000 7fc3348ab700 -1 osd.0 0 waiting for initial osdmap 2020-11-20T13:56:12.895 INFO:tasks.ceph.osd.7.smithi116.stderr:2020-11-20T13:56:12.892+0000 7f690eeb8700 -1 osd.7 0 waiting for initial osdmap 2020-11-20T13:56:12.897 INFO:tasks.ceph.osd.6.smithi116.stderr:2020-11-20T13:56:12.896+0000 7f0b9c75a700 -1 osd.6 0 waiting for initial osdmap 2020-11-20T13:56:12.897 INFO:tasks.ceph.osd.3.smithi040.stderr:2020-11-20T13:56:12.896+0000 7f407c596700 -1 osd.3 0 waiting for initial osdmap 2020-11-20T13:56:12.898 INFO:tasks.ceph.osd.2.smithi040.stderr:2020-11-20T13:56:12.896+0000 7fdd2a167700 -1 osd.2 0 waiting for initial osdmap 2020-11-20T13:56:12.898 INFO:tasks.ceph.osd.1.smithi040.stderr:2020-11-20T13:56:12.897+0000 7f26fc0d9700 -1 osd.1 0 waiting for initial osdmap 2020-11-20T13:56:12.898 INFO:tasks.ceph.osd.4.smithi116.stderr:2020-11-20T13:56:12.897+0000 7fc87c9ce700 -1 osd.4 0 waiting for initial osdmap 2020-11-20T13:56:12.899 INFO:tasks.ceph.osd.5.smithi116.stderr:2020-11-20T13:56:12.897+0000 7fc8a4cd1700 -1 osd.5 0 waiting for initial osdmap 2020-11-20T13:56:17.326 INFO:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2020-11-20T13:56:17.440 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.436+0000 7f87beca6700 1 Processor -- start 2020-11-20T13:56:17.440 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.436+0000 7f87beca6700 1 -- start start 2020-11-20T13:56:17.440 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.437+0000 7f87beca6700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f87b805f710 0x7f87b8067350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:17.441 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.437+0000 7f87beca6700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f87b814d9a0 0x7f87b814de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:17.441 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.437+0000 7f87beca6700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b8080430 con 0x7f87b8157330 2020-11-20T13:56:17.441 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.437+0000 7f87beca6700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87b814e310 con 0x7f87b814d9a0 2020-11-20T13:56:17.441 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.437+0000 7f87beca6700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b80806d0 con 0x7f87b805f710 2020-11-20T13:56:17.441 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.438+0000 7f87bd243700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f87b805f710 0x7f87b8067350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:17.442 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.438+0000 7f87bca42700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f87b814d9a0 0x7f87b814de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:17.442 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.438+0000 7f87bd243700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f87b805f710 0x7f87b8067350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.40:39370/0 (socket says 172.21.15.40:39370) 2020-11-20T13:56:17.442 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.438+0000 7f87bd243700 1 -- 172.21.15.40:0/3781717226 learned_addr learned my addr 172.21.15.40:0/3781717226 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:17.442 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.438+0000 7f87bca42700 1 -- 172.21.15.40:0/3781717226 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f87b805f710 msgr2=0x7f87b8067350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:17.442 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.438+0000 7f87bca42700 1 --2- 172.21.15.40:0/3781717226 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f87b805f710 0x7f87b8067350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.443 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.438+0000 7f87bca42700 1 -- 172.21.15.40:0/3781717226 >> v1:172.21.15.116:6789/0 conn(0x7f87b8157330 legacy=0x7f87b81577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:17.443 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.438+0000 7f87bca42700 1 -- 172.21.15.40:0/3781717226 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b805faf0 con 0x7f87b814d9a0 2020-11-20T13:56:17.443 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.438+0000 7f87affff700 1 -- 172.21.15.40:0/3781717226 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1979237446 0 0) 0x7f87b8080430 con 0x7f87b8157330 2020-11-20T13:56:17.443 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.438+0000 7f87bca42700 1 --2- 172.21.15.40:0/3781717226 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87b814d9a0 0x7f87b814de20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f87a8002f30 tx=0x7f87a8006dc0).ready entity=mon.1 client_cookie=62a77478b50301c5 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:17.444 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.438+0000 7f87affff700 1 -- 172.21.15.40:0/3781717226 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87a8009760 con 0x7f87b814d9a0 2020-11-20T13:56:17.444 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.438+0000 7f87affff700 1 -- 172.21.15.40:0/3781717226 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a80098c0 con 0x7f87b814d9a0 2020-11-20T13:56:17.444 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.438+0000 7f87affff700 1 -- 172.21.15.40:0/3781717226 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87a8009c20 con 0x7f87b814d9a0 2020-11-20T13:56:17.444 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.438+0000 7f87beca6700 1 -- 172.21.15.40:0/3781717226 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87b814d9a0 msgr2=0x7f87b814de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:17.444 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.438+0000 7f87beca6700 1 --2- 172.21.15.40:0/3781717226 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87b814d9a0 0x7f87b814de20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f87a8002f30 tx=0x7f87a8006dc0).stop 2020-11-20T13:56:17.445 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.439+0000 7f87beca6700 1 -- 172.21.15.40:0/3781717226 shutdown_connections 2020-11-20T13:56:17.445 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.439+0000 7f87beca6700 1 --2- 172.21.15.40:0/3781717226 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f87b805f710 0x7f87b8067350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.445 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.439+0000 7f87beca6700 1 --2- 172.21.15.40:0/3781717226 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87b814d9a0 0x7f87b814de20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.445 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.439+0000 7f87beca6700 1 -- 172.21.15.40:0/3781717226 >> 172.21.15.40:0/3781717226 conn(0x7f87b805c1c0 msgr2=0x7f87b805c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:17.446 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.439+0000 7f87beca6700 1 -- 172.21.15.40:0/3781717226 shutdown_connections 2020-11-20T13:56:17.446 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.439+0000 7f87beca6700 1 -- 172.21.15.40:0/3781717226 wait complete. 2020-11-20T13:56:17.449 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.440+0000 7f87beca6700 1 Processor -- start 2020-11-20T13:56:17.450 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.440+0000 7f87beca6700 1 -- start start 2020-11-20T13:56:17.450 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.441+0000 7f87beca6700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87b805f710 0x7f87b80b7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:17.450 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.441+0000 7f87beca6700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f87b814d9a0 0x7f87b8157740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:17.450 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.441+0000 7f87beca6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87b8157330 0x7f87b80ad720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:17.450 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.441+0000 7f87beca6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b80b7910 con 0x7f87b8157330 2020-11-20T13:56:17.451 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.441+0000 7f87beca6700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b80b7b30 con 0x7f87b805f710 2020-11-20T13:56:17.451 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.441+0000 7f87beca6700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b80b7dd0 con 0x7f87b814d9a0 2020-11-20T13:56:17.451 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.441+0000 7f87bda44700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87b8157330 0x7f87b80ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:17.451 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.441+0000 7f87bda44700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87b8157330 0x7f87b80ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54304/0 (socket says 172.21.15.40:54304) 2020-11-20T13:56:17.451 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.441+0000 7f87bda44700 1 -- 172.21.15.40:0/1006322336 learned_addr learned my addr 172.21.15.40:0/1006322336 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:17.452 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.441+0000 7f87bca42700 1 --2- 172.21.15.40:0/1006322336 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f87b814d9a0 0x7f87b8157740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:17.452 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.441+0000 7f87bd243700 1 --2- 172.21.15.40:0/1006322336 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87b805f710 0x7f87b80b7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:17.452 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.442+0000 7f87bda44700 1 -- 172.21.15.40:0/1006322336 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f87b814d9a0 msgr2=0x7f87b8157740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:17.452 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.442+0000 7f87bda44700 1 --2- 172.21.15.40:0/1006322336 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f87b814d9a0 0x7f87b8157740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.452 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.442+0000 7f87bda44700 1 -- 172.21.15.40:0/1006322336 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87b805f710 msgr2=0x7f87b80b7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:17.453 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.442+0000 7f87bda44700 1 --2- 172.21.15.40:0/1006322336 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87b805f710 0x7f87b80b7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.453 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.442+0000 7f87bda44700 1 -- 172.21.15.40:0/1006322336 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b80b8070 con 0x7f87b8157330 2020-11-20T13:56:17.453 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.442+0000 7f87bca42700 1 --2- 172.21.15.40:0/1006322336 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f87b814d9a0 0x7f87b8157740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:56:17.453 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.442+0000 7f87bd243700 1 --2- 172.21.15.40:0/1006322336 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87b805f710 0x7f87b80b7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:56:17.453 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.442+0000 7f87bda44700 1 --2- 172.21.15.40:0/1006322336 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87b8157330 0x7f87b80ad720 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f87b0001210 tx=0x7f87b0007d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:17.454 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.442+0000 7f87adffb700 1 -- 172.21.15.40:0/1006322336 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87b000ce60 con 0x7f87b8157330 2020-11-20T13:56:17.454 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.442+0000 7f87adffb700 1 -- 172.21.15.40:0/1006322336 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b000a7b0 con 0x7f87b8157330 2020-11-20T13:56:17.454 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.442+0000 7f87beca6700 1 -- 172.21.15.40:0/1006322336 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b814d580 con 0x7f87b8157330 2020-11-20T13:56:17.454 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.442+0000 7f87adffb700 1 -- 172.21.15.40:0/1006322336 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87b000ab10 con 0x7f87b8157330 2020-11-20T13:56:17.454 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.442+0000 7f87beca6700 1 -- 172.21.15.40:0/1006322336 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b8157e70 con 0x7f87b8157330 2020-11-20T13:56:17.455 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.443+0000 7f87beca6700 1 -- 172.21.15.40:0/1006322336 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b80b8200 con 0x7f87b8157330 2020-11-20T13:56:17.455 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.443+0000 7f87adffb700 1 -- 172.21.15.40:0/1006322336 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f87b0019030 con 0x7f87b8157330 2020-11-20T13:56:17.455 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.443+0000 7f87adffb700 1 --2- 172.21.15.40:0/1006322336 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f8798051290 0x7f8798053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:17.455 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.443+0000 7f87adffb700 1 -- 172.21.15.40:0/1006322336 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4900+0+0 (secure 0 0 0) 0x7f87b00680a0 con 0x7f87b8157330 2020-11-20T13:56:17.455 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.444+0000 7f87bd243700 1 --2- 172.21.15.40:0/1006322336 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f8798051290 0x7f8798053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:17.456 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.444+0000 7f87bd243700 1 --2- 172.21.15.40:0/1006322336 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f8798051290 0x7f8798053750 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f87b4008000 tx=0x7f87b4008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:17.456 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.447+0000 7f87adffb700 1 -- 172.21.15.40:0/1006322336 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f87b0040960 con 0x7f87b8157330 2020-11-20T13:56:17.606 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.606+0000 7f87beca6700 1 -- 172.21.15.40:0/1006322336 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f87b8057050 con 0x7f87b8157330 2020-11-20T13:56:17.607 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.607+0000 7f87adffb700 1 -- 172.21.15.40:0/1006322336 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+11853 (secure 0 0 0) 0x7f87b0023e60 con 0x7f87b8157330 2020-11-20T13:56:17.608 INFO:teuthology.misc.health.smithi040.stdout: 2020-11-20T13:56:17.608 INFO:teuthology.misc.health.smithi040.stdout:{"epoch":15,"fsid":"da1ed737-5174-430f-a3c9-56f6bbfad13c","created":"2020-11-20T13:56:04.990651+0000","modified":"2020-11-20T13:56:15.910192+0000","last_up_change":"2020-11-20T13:56:13.893026+0000","last_in_change":"2020-11-20T13:56:13.893026+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T13:56:08.330017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"9ea4c667-8938-46d6-a5ca-092d55c1f79e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":22825},{"type":"v1","addr":"172.21.15.40:6818","nonce":22825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":22825},{"type":"v1","addr":"172.21.15.40:6822","nonce":22825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":22825},{"type":"v1","addr":"172.21.15.40:6830","nonce":22825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":22825},{"type":"v1","addr":"172.21.15.40:6826","nonce":22825}]},"public_addr":"172.21.15.40:6818/22825","cluster_addr":"172.21.15.40:6822/22825","heartbeat_back_addr":"172.21.15.40:6830/22825","heartbeat_front_addr":"172.21.15.40:6826/22825","state":["exists","up"]},{"osd":1,"uuid":"515708d9-5087-4e55-8d14-cdea1c00eacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":22823},{"type":"v1","addr":"172.21.15.40:6801","nonce":22823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":22823},{"type":"v1","addr":"172.21.15.40:6803","nonce":22823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":22823},{"type":"v1","addr":"172.21.15.40:6807","nonce":22823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":22823},{"type":"v1","addr":"172.21.15.40:6805","nonce":22823}]},"public_addr":"172.21.15.40:6801/22823","cluster_addr":"172.21.15.40:6803/22823","heartbeat_back_addr":"172.21.15.40:6807/22823","heartbeat_front_addr":"172.21.15.40:6805/22823","state":["exists","up"]},{"osd":2,"uuid":"4d86bca7-743e-4983-b35f-36676c405b36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":22824},{"type":"v1","addr":"172.21.15.40:6809","nonce":22824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":22824},{"type":"v1","addr":"172.21.15.40:6811","nonce":22824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":22824},{"type":"v1","addr":"172.21.15.40:6815","nonce":22824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":22824},{"type":"v1","addr":"172.21.15.40:6813","nonce":22824}]},"public_addr":"172.21.15.40:6809/22824","cluster_addr":"172.21.15.40:6811/22824","heartbeat_back_addr":"172.21.15.40:6815/22824","heartbeat_front_addr":"172.21.15.40:6813/22824","state":["exists","up"]},{"osd":3,"uuid":"f4dc726b-6b34-4e8b-8eeb-87060b58002d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":22826},{"type":"v1","addr":"172.21.15.40:6819","nonce":22826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":22826},{"type":"v1","addr":"172.21.15.40:6823","nonce":22826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":22826},{"type":"v1","addr":"172.21.15.40:6831","nonce":22826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":22826},{"type":"v1","addr":"172.21.15.40:6827","nonce":22826}]},"public_addr":"172.21.15.40:6819/22826","cluster_addr":"172.21.15.40:6823/22826","heartbeat_back_addr":"172.21.15.40:6831/22826","heartbeat_front_addr":"172.21.15.40:6827/22826","state":["exists","up"]},{"osd":4,"uuid":"25a6d53b-d1f0-4498-b7d5-ffa38661d403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":23181},{"type":"v1","addr":"172.21.15.116:6819","nonce":23181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":23181},{"type":"v1","addr":"172.21.15.116:6821","nonce":23181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":23181},{"type":"v1","addr":"172.21.15.116:6825","nonce":23181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":23181},{"type":"v1","addr":"172.21.15.116:6823","nonce":23181}]},"public_addr":"172.21.15.116:6819/23181","cluster_addr":"172.21.15.116:6821/23181","heartbeat_back_addr":"172.21.15.116:6825/23181","heartbeat_front_addr":"172.21.15.116:6823/23181","state":["exists","up"]},{"osd":5,"uuid":"7d7a5af8-229b-4351-9a21-a3d509cd4635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":23171},{"type":"v1","addr":"172.21.15.116:6803","nonce":23171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":23171},{"type":"v1","addr":"172.21.15.116:6805","nonce":23171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":23171},{"type":"v1","addr":"172.21.15.116:6809","nonce":23171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":23171},{"type":"v1","addr":"172.21.15.116:6807","nonce":23171}]},"public_addr":"172.21.15.116:6803/23171","cluster_addr":"172.21.15.116:6805/23171","heartbeat_back_addr":"172.21.15.116:6809/23171","heartbeat_front_addr":"172.21.15.116:6807/23171","state":["exists","up"]},{"osd":6,"uuid":"8247d248-a140-4e36-9796-9847a97df58d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":23172},{"type":"v1","addr":"172.21.15.116:6811","nonce":23172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":23172},{"type":"v1","addr":"172.21.15.116:6813","nonce":23172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":23172},{"type":"v1","addr":"172.21.15.116:6817","nonce":23172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":23172},{"type":"v1","addr":"172.21.15.116:6815","nonce":23172}]},"public_addr":"172.21.15.116:6811/23172","cluster_addr":"172.21.15.116:6813/23172","heartbeat_back_addr":"172.21.15.116:6817/23172","heartbeat_front_addr":"172.21.15.116:6815/23172","state":["exists","up"]},{"osd":7,"uuid":"472803c4-70b8-499d-b18a-71aaeec6aab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":23196},{"type":"v1","addr":"172.21.15.116:6827","nonce":23196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":23196},{"type":"v1","addr":"172.21.15.116:6829","nonce":23196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":23196},{"type":"v1","addr":"172.21.15.116:6833","nonce":23196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":23196},{"type":"v1","addr":"172.21.15.116:6831","nonce":23196}]},"public_addr":"172.21.15.116:6827/23196","cluster_addr":"172.21.15.116:6829/23196","heartbeat_back_addr":"172.21.15.116:6833/23196","heartbeat_front_addr":"172.21.15.116:6831/23196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.029357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:11.989797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.091097+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.014766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.070860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.071864+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.063400+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.109028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-20T13:56:17.609 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.608+0000 7f87beca6700 1 -- 172.21.15.40:0/1006322336 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f8798051290 msgr2=0x7f8798053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:17.609 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.608+0000 7f87beca6700 1 --2- 172.21.15.40:0/1006322336 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f8798051290 0x7f8798053750 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f87b4008000 tx=0x7f87b4008040).stop 2020-11-20T13:56:17.610 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.609+0000 7f87beca6700 1 -- 172.21.15.40:0/1006322336 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87b8157330 msgr2=0x7f87b80ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:17.610 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.609+0000 7f87beca6700 1 --2- 172.21.15.40:0/1006322336 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87b8157330 0x7f87b80ad720 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f87b0001210 tx=0x7f87b0007d10).stop 2020-11-20T13:56:17.610 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.609+0000 7f87beca6700 1 -- 172.21.15.40:0/1006322336 shutdown_connections 2020-11-20T13:56:17.610 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.609+0000 7f87beca6700 1 --2- 172.21.15.40:0/1006322336 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f8798051290 0x7f8798053750 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.611 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.609+0000 7f87beca6700 1 --2- 172.21.15.40:0/1006322336 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87b8157330 0x7f87b80ad720 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.611 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.609+0000 7f87beca6700 1 --2- 172.21.15.40:0/1006322336 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87b805f710 0x7f87b80b7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.611 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.609+0000 7f87beca6700 1 --2- 172.21.15.40:0/1006322336 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f87b814d9a0 0x7f87b8157740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.611 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.609+0000 7f87beca6700 1 -- 172.21.15.40:0/1006322336 >> 172.21.15.40:0/1006322336 conn(0x7f87b805c1c0 msgr2=0x7f87b8150d30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:17.611 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.609+0000 7f87beca6700 1 -- 172.21.15.40:0/1006322336 shutdown_connections 2020-11-20T13:56:17.612 INFO:teuthology.misc.health.smithi040.stderr:2020-11-20T13:56:17.609+0000 7f87beca6700 1 -- 172.21.15.40:0/1006322336 wait complete. 2020-11-20T13:56:17.618 DEBUG:teuthology.misc:8 of 8 OSDs are up 2020-11-20T13:56:17.619 INFO:tasks.ceph:Creating RBD pool 2020-11-20T13:56:17.619 INFO:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph osd pool create rbd 8 2020-11-20T13:56:17.737 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.732+0000 7f94a8464700 1 Processor -- start 2020-11-20T13:56:17.737 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.733+0000 7f94a8464700 1 -- start start 2020-11-20T13:56:17.737 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.733+0000 7f94a8464700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f94a0157bf0 0x7f94a00b3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:17.737 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.733+0000 7f94a8464700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94a01574b0 con 0x7f94a0158070 2020-11-20T13:56:17.738 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.733+0000 7f94a8464700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94a0157750 con 0x7f94a00805f0 2020-11-20T13:56:17.738 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.733+0000 7f94a8464700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94a00809d0 con 0x7f94a0157bf0 2020-11-20T13:56:17.738 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.734+0000 7f94a6200700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f94a0157bf0 0x7f94a00b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:17.738 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.734+0000 7f94a6200700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f94a0157bf0 0x7f94a00b3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54312/0 (socket says 172.21.15.40:54312) 2020-11-20T13:56:17.739 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.734+0000 7f94a6200700 1 -- 172.21.15.40:0/3788931953 learned_addr learned my addr 172.21.15.40:0/3788931953 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:17.739 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.734+0000 7f94a6200700 1 -- 172.21.15.40:0/3788931953 >> v1:172.21.15.116:6789/0 conn(0x7f94a00805f0 legacy=0x7f94a00b5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:17.739 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.734+0000 7f94a6200700 1 -- 172.21.15.40:0/3788931953 >> v1:172.21.15.40:6789/0 conn(0x7f94a0158070 legacy=0x7f94a0153e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:17.739 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.734+0000 7f94a6200700 1 -- 172.21.15.40:0/3788931953 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a00b6820 con 0x7f94a0157bf0 2020-11-20T13:56:17.739 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.734+0000 7f94a59ff700 1 -- 172.21.15.40:0/3788931953 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1183363060 0 0) 0x7f94a01574b0 con 0x7f94a0158070 2020-11-20T13:56:17.740 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.734+0000 7f94a6200700 1 --2- 172.21.15.40:0/3788931953 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a0157bf0 0x7f94a00b3010 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f9494002f30 tx=0x7f9494006e80).ready entity=mon.0 client_cookie=ae21e27dacb47095 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:17.740 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.734+0000 7f94a59ff700 1 -- 172.21.15.40:0/3788931953 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94940097f0 con 0x7f94a0157bf0 2020-11-20T13:56:17.740 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.734+0000 7f94a59ff700 1 -- 172.21.15.40:0/3788931953 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9494009950 con 0x7f94a0157bf0 2020-11-20T13:56:17.740 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.734+0000 7f94a59ff700 1 -- 172.21.15.40:0/3788931953 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9494009cb0 con 0x7f94a0157bf0 2020-11-20T13:56:17.740 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.734+0000 7f94a8464700 1 -- 172.21.15.40:0/3788931953 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a0157bf0 msgr2=0x7f94a00b3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:17.741 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.734+0000 7f94a8464700 1 --2- 172.21.15.40:0/3788931953 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a0157bf0 0x7f94a00b3010 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f9494002f30 tx=0x7f9494006e80).stop 2020-11-20T13:56:17.741 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.735+0000 7f94a8464700 1 -- 172.21.15.40:0/3788931953 shutdown_connections 2020-11-20T13:56:17.741 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.735+0000 7f94a8464700 1 --2- 172.21.15.40:0/3788931953 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a0157bf0 0x7f94a00b3010 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.741 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.735+0000 7f94a8464700 1 -- 172.21.15.40:0/3788931953 >> 172.21.15.40:0/3788931953 conn(0x7f94a005f390 msgr2=0x7f94a005f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:17.741 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.735+0000 7f94a8464700 1 -- 172.21.15.40:0/3788931953 shutdown_connections 2020-11-20T13:56:17.742 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.735+0000 7f94a8464700 1 -- 172.21.15.40:0/3788931953 wait complete. 2020-11-20T13:56:17.742 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.736+0000 7f94a8464700 1 Processor -- start 2020-11-20T13:56:17.742 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.737+0000 7f94a8464700 1 -- start start 2020-11-20T13:56:17.745 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.737+0000 7f94a8464700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a00805f0 0x7f94a0153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:17.746 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.737+0000 7f94a8464700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f94a0157bf0 0x7f94a0134090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:17.746 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.737+0000 7f94a8464700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94a0158070 0x7f94a00ad7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:17.746 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.737+0000 7f94a8464700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a0062d70 con 0x7f94a00805f0 2020-11-20T13:56:17.748 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.737+0000 7f94a8464700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a0062f60 con 0x7f94a0158070 2020-11-20T13:56:17.748 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.737+0000 7f94a8464700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a0063200 con 0x7f94a0157bf0 2020-11-20T13:56:17.748 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.737+0000 7f94a6a01700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a00805f0 0x7f94a0153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:17.748 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.737+0000 7f94a6a01700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a00805f0 0x7f94a0153e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54316/0 (socket says 172.21.15.40:54316) 2020-11-20T13:56:17.748 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.737+0000 7f94a6a01700 1 -- 172.21.15.40:0/116732123 learned_addr learned my addr 172.21.15.40:0/116732123 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:17.749 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.737+0000 7f94a7202700 1 --2- 172.21.15.40:0/116732123 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94a0158070 0x7f94a00ad7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:17.749 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.737+0000 7f94a6200700 1 --2- 172.21.15.40:0/116732123 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f94a0157bf0 0x7f94a0134090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:17.749 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.738+0000 7f94a6a01700 1 -- 172.21.15.40:0/116732123 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f94a0157bf0 msgr2=0x7f94a0134090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:17.749 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.738+0000 7f94a6a01700 1 --2- 172.21.15.40:0/116732123 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f94a0157bf0 0x7f94a0134090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.750 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.738+0000 7f94a6a01700 1 -- 172.21.15.40:0/116732123 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94a0158070 msgr2=0x7f94a00ad7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:17.750 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.738+0000 7f94a6a01700 1 --2- 172.21.15.40:0/116732123 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94a0158070 0x7f94a00ad7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.750 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.738+0000 7f94a6a01700 1 -- 172.21.15.40:0/116732123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a00636a0 con 0x7f94a00805f0 2020-11-20T13:56:17.750 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.738+0000 7f94a7202700 1 --2- 172.21.15.40:0/116732123 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94a0158070 0x7f94a00ad7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:17.750 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.738+0000 7f94a6a01700 1 --2- 172.21.15.40:0/116732123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a00805f0 0x7f94a0153e40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f9490007800 tx=0x7f9490007b90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:17.751 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.738+0000 7f949f7fe700 1 -- 172.21.15.40:0/116732123 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94900108a0 con 0x7f94a00805f0 2020-11-20T13:56:17.751 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.738+0000 7f949f7fe700 1 -- 172.21.15.40:0/116732123 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9490010a00 con 0x7f94a00805f0 2020-11-20T13:56:17.751 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.738+0000 7f94a8464700 1 -- 172.21.15.40:0/116732123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a0063890 con 0x7f94a00805f0 2020-11-20T13:56:17.751 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.738+0000 7f949f7fe700 1 -- 172.21.15.40:0/116732123 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9490010d60 con 0x7f94a00805f0 2020-11-20T13:56:17.751 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.738+0000 7f94a8464700 1 -- 172.21.15.40:0/116732123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94a0063dd0 con 0x7f94a00805f0 2020-11-20T13:56:17.752 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.739+0000 7f949f7fe700 1 -- 172.21.15.40:0/116732123 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f9490011470 con 0x7f94a00805f0 2020-11-20T13:56:17.752 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.739+0000 7f94a8464700 1 -- 172.21.15.40:0/116732123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94a00633c0 con 0x7f94a00805f0 2020-11-20T13:56:17.752 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.739+0000 7f949f7fe700 1 --2- 172.21.15.40:0/116732123 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f9480051320 0x7f94800537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:17.752 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.739+0000 7f949f7fe700 1 -- 172.21.15.40:0/116732123 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4900+0+0 (secure 0 0 0) 0x7f949006aca0 con 0x7f94a00805f0 2020-11-20T13:56:17.752 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.739+0000 7f94a6200700 1 --2- 172.21.15.40:0/116732123 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f9480051320 0x7f94800537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:17.753 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.740+0000 7f94a6200700 1 --2- 172.21.15.40:0/116732123 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f9480051320 0x7f94800537e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f949401a000 tx=0x7f9494015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:17.753 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.743+0000 7f949f7fe700 1 -- 172.21.15.40:0/116732123 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f949006f070 con 0x7f94a00805f0 2020-11-20T13:56:17.904 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.904+0000 7f94a8464700 1 -- 172.21.15.40:0/116732123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f94a0057050 con 0x7f94a00805f0 2020-11-20T13:56:17.994 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.994+0000 7f949f7fe700 1 -- 172.21.15.40:0/116732123 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v16) v1 ==== 109+0+0 (secure 0 0 0) 0x7f9490043450 con 0x7f94a00805f0 2020-11-20T13:56:17.997 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.996+0000 7f94a8464700 1 -- 172.21.15.40:0/116732123 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f9480051320 msgr2=0x7f94800537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:17.997 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.996+0000 7f94a8464700 1 --2- 172.21.15.40:0/116732123 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f9480051320 0x7f94800537e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f949401a000 tx=0x7f9494015000).stop 2020-11-20T13:56:17.997 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.997+0000 7f94a8464700 1 -- 172.21.15.40:0/116732123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a00805f0 msgr2=0x7f94a0153e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:17.998 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.997+0000 7f94a8464700 1 --2- 172.21.15.40:0/116732123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a00805f0 0x7f94a0153e40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f9490007800 tx=0x7f9490007b90).stop 2020-11-20T13:56:17.998 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.997+0000 7f94a8464700 1 -- 172.21.15.40:0/116732123 shutdown_connections 2020-11-20T13:56:17.998 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.997+0000 7f94a8464700 1 --2- 172.21.15.40:0/116732123 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f9480051320 0x7f94800537e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.998 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.997+0000 7f94a8464700 1 --2- 172.21.15.40:0/116732123 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94a0158070 0x7f94a00ad7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.998 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.997+0000 7f94a8464700 1 --2- 172.21.15.40:0/116732123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a00805f0 0x7f94a0153e40 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.999 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.997+0000 7f94a8464700 1 --2- 172.21.15.40:0/116732123 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f94a0157bf0 0x7f94a0134090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.999 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.997+0000 7f94a8464700 1 -- 172.21.15.40:0/116732123 >> 172.21.15.40:0/116732123 conn(0x7f94a005f390 msgr2=0x7f94a00b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:17.999 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.997+0000 7f94a8464700 1 -- 172.21.15.40:0/116732123 shutdown_connections 2020-11-20T13:56:17.999 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:17.997+0000 7f94a8464700 1 -- 172.21.15.40:0/116732123 wait complete. 2020-11-20T13:56:18.000 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'rbd' created 2020-11-20T13:56:18.010 INFO:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2020-11-20T13:56:18.128 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.125+0000 7fbcc119c700 1 Processor -- start 2020-11-20T13:56:18.128 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.125+0000 7fbcc119c700 1 -- start start 2020-11-20T13:56:18.129 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.126+0000 7fbcc119c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fbcbc0805f0 0x7fbcbc0b5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:18.129 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.126+0000 7fbcc119c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbcbc0b3010 0x7fbcbc0b3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:18.129 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.126+0000 7fbcc119c700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcbc157750 con 0x7fbcbc0b6390 2020-11-20T13:56:18.129 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.126+0000 7fbcc119c700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcbc157540 con 0x7fbcbc0b3010 2020-11-20T13:56:18.129 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.126+0000 7fbcc119c700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcbc0809d0 con 0x7fbcbc0805f0 2020-11-20T13:56:18.130 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.126+0000 7fbcc099b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbcbc0b3010 0x7fbcbc0b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:18.130 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.126+0000 7fbcc099b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbcbc0b3010 0x7fbcbc0b3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54328/0 (socket says 172.21.15.40:54328) 2020-11-20T13:56:18.130 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.126+0000 7fbcc099b700 1 -- 172.21.15.40:0/515996703 learned_addr learned my addr 172.21.15.40:0/515996703 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:18.130 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.126+0000 7fbcbb59e700 1 --2- 172.21.15.40:0/515996703 >> v2:172.21.15.116:3300/0 conn(0x7fbcbc0805f0 0x7fbcbc0b5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:18.130 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.126+0000 7fbcc099b700 1 -- 172.21.15.40:0/515996703 >> v2:172.21.15.116:3300/0 conn(0x7fbcbc0805f0 msgr2=0x7fbcbc0b5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:18.131 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.126+0000 7fbcc099b700 1 --2- 172.21.15.40:0/515996703 >> v2:172.21.15.116:3300/0 conn(0x7fbcbc0805f0 0x7fbcbc0b5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:18.131 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.126+0000 7fbcc099b700 1 -- 172.21.15.40:0/515996703 >> v1:172.21.15.116:6789/0 conn(0x7fbcbc0b6390 legacy=0x7fbcbc157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:18.131 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.126+0000 7fbcc099b700 1 -- 172.21.15.40:0/515996703 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcbc158300 con 0x7fbcbc0b3010 2020-11-20T13:56:18.132 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.127+0000 7fbcc099b700 1 --2- 172.21.15.40:0/515996703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcbc0b3010 0x7fbcbc0b3480 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fbcb40032b0 tx=0x7fbcb4009270).ready entity=mon.0 client_cookie=89bbb701deeb4567 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:18.132 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.127+0000 7fbcba59c700 1 -- 172.21.15.40:0/515996703 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbcb400bc00 con 0x7fbcbc0b3010 2020-11-20T13:56:18.132 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.127+0000 7fbcba59c700 1 -- 172.21.15.40:0/515996703 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcb400bd60 con 0x7fbcbc0b3010 2020-11-20T13:56:18.132 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.127+0000 7fbcba59c700 1 -- 172.21.15.40:0/515996703 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbcb4011400 con 0x7fbcbc0b3010 2020-11-20T13:56:18.133 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.127+0000 7fbcc119c700 1 -- 172.21.15.40:0/515996703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcbc0b3010 msgr2=0x7fbcbc0b3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:18.133 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.127+0000 7fbcc119c700 1 --2- 172.21.15.40:0/515996703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcbc0b3010 0x7fbcbc0b3480 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fbcb40032b0 tx=0x7fbcb4009270).stop 2020-11-20T13:56:18.133 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.127+0000 7fbcc119c700 1 -- 172.21.15.40:0/515996703 shutdown_connections 2020-11-20T13:56:18.133 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.127+0000 7fbcc119c700 1 --2- 172.21.15.40:0/515996703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcbc0b3010 0x7fbcbc0b3480 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:18.133 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.127+0000 7fbcc119c700 1 --2- 172.21.15.40:0/515996703 >> v2:172.21.15.116:3300/0 conn(0x7fbcbc0805f0 0x7fbcbc0b5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:18.134 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.127+0000 7fbcc119c700 1 -- 172.21.15.40:0/515996703 >> 172.21.15.40:0/515996703 conn(0x7fbcbc05f390 msgr2=0x7fbcbc05f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:18.134 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.127+0000 7fbcc119c700 1 -- 172.21.15.40:0/515996703 shutdown_connections 2020-11-20T13:56:18.134 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.127+0000 7fbcc119c700 1 -- 172.21.15.40:0/515996703 wait complete. 2020-11-20T13:56:18.136 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.128+0000 7fbcc119c700 1 Processor -- start 2020-11-20T13:56:18.137 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.129+0000 7fbcc119c700 1 -- start start 2020-11-20T13:56:18.138 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.129+0000 7fbcc119c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbcbc0805f0 0x7fbcbc150790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:18.138 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.129+0000 7fbcc119c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcbc0b3010 0x7fbcbc0b0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:18.138 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.129+0000 7fbcc119c700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbcbc0667b0 0x7fbcbc13e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:18.138 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.129+0000 7fbcc119c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc1500c0 con 0x7fbcbc0b3010 2020-11-20T13:56:18.138 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.129+0000 7fbcc119c700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc063210 con 0x7fbcbc0805f0 2020-11-20T13:56:18.139 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.129+0000 7fbcc119c700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc066ca0 con 0x7fbcbc0667b0 2020-11-20T13:56:18.139 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.129+0000 7fbcbad9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcbc0b3010 0x7fbcbc0b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:18.139 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.129+0000 7fbcbad9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcbc0b3010 0x7fbcbc0b0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54334/0 (socket says 172.21.15.40:54334) 2020-11-20T13:56:18.139 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.129+0000 7fbcbad9d700 1 -- 172.21.15.40:0/1663213526 learned_addr learned my addr 172.21.15.40:0/1663213526 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:18.139 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.130+0000 7fbcbb59e700 1 --2- 172.21.15.40:0/1663213526 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbcbc0805f0 0x7fbcbc150790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:18.140 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.130+0000 7fbcc099b700 1 --2- 172.21.15.40:0/1663213526 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbcbc0667b0 0x7fbcbc13e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:18.140 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.130+0000 7fbcbad9d700 1 -- 172.21.15.40:0/1663213526 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbcbc0667b0 msgr2=0x7fbcbc13e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:18.140 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.130+0000 7fbcbad9d700 1 --2- 172.21.15.40:0/1663213526 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbcbc0667b0 0x7fbcbc13e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:18.140 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.130+0000 7fbcbad9d700 1 -- 172.21.15.40:0/1663213526 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbcbc0805f0 msgr2=0x7fbcbc150790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:18.140 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.130+0000 7fbcbad9d700 1 --2- 172.21.15.40:0/1663213526 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbcbc0805f0 0x7fbcbc150790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:18.141 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.130+0000 7fbcbad9d700 1 -- 172.21.15.40:0/1663213526 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcbc1502f0 con 0x7fbcbc0b3010 2020-11-20T13:56:18.141 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.130+0000 7fbcbad9d700 1 --2- 172.21.15.40:0/1663213526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcbc0b3010 0x7fbcbc0b0620 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7fbcbc067d50 tx=0x7fbcb0007b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:18.141 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.130+0000 7fbca3fff700 1 -- 172.21.15.40:0/1663213526 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbcb0007030 con 0x7fbcbc0b3010 2020-11-20T13:56:18.141 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.130+0000 7fbca3fff700 1 -- 172.21.15.40:0/1663213526 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcb0007190 con 0x7fbcbc0b3010 2020-11-20T13:56:18.142 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.130+0000 7fbcc119c700 1 -- 172.21.15.40:0/1663213526 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcbc062670 con 0x7fbcbc0b3010 2020-11-20T13:56:18.142 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.130+0000 7fbca3fff700 1 -- 172.21.15.40:0/1663213526 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbcb00064e0 con 0x7fbcbc0b3010 2020-11-20T13:56:18.142 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.130+0000 7fbcc119c700 1 -- 172.21.15.40:0/1663213526 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcbc0b0de0 con 0x7fbcbc0b3010 2020-11-20T13:56:18.142 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.131+0000 7fbcc119c700 1 -- 172.21.15.40:0/1663213526 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcbc057050 con 0x7fbcbc0b3010 2020-11-20T13:56:18.142 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.131+0000 7fbca3fff700 1 -- 172.21.15.40:0/1663213526 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fbcb0010070 con 0x7fbcbc0b3010 2020-11-20T13:56:18.143 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.132+0000 7fbca3fff700 1 --2- 172.21.15.40:0/1663213526 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fbc9c051290 0x7fbc9c053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:18.143 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.132+0000 7fbca3fff700 1 -- 172.21.15.40:0/1663213526 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5242+0+0 (secure 0 0 0) 0x7fbcb0068180 con 0x7fbcbc0b3010 2020-11-20T13:56:18.143 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.132+0000 7fbcbb59e700 1 --2- 172.21.15.40:0/1663213526 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fbc9c051290 0x7fbc9c053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:18.143 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.132+0000 7fbcbb59e700 1 --2- 172.21.15.40:0/1663213526 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fbc9c051290 0x7fbc9c053750 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fbcac00c000 tx=0x7fbcac008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:18.143 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.135+0000 7fbca3fff700 1 -- 172.21.15.40:0/1663213526 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fbcb00407e0 con 0x7fbcbc0b3010 2020-11-20T13:56:18.303 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:18.303+0000 7fbcc119c700 1 -- 172.21.15.40:0/1663213526 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbcbc063d20 con 0x7fbcbc0b3010 2020-11-20T13:56:19.000 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.000+0000 7fbca3fff700 1 -- 172.21.15.40:0/1663213526 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v17) v1 ==== 173+0+0 (secure 0 0 0) 0x7fbcb0001e60 con 0x7fbcbc0b3010 2020-11-20T13:56:19.002 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.002+0000 7fbcc119c700 1 -- 172.21.15.40:0/1663213526 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fbc9c051290 msgr2=0x7fbc9c053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:19.003 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.002+0000 7fbcc119c700 1 --2- 172.21.15.40:0/1663213526 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fbc9c051290 0x7fbc9c053750 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fbcac00c000 tx=0x7fbcac008040).stop 2020-11-20T13:56:19.003 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.002+0000 7fbcc119c700 1 -- 172.21.15.40:0/1663213526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcbc0b3010 msgr2=0x7fbcbc0b0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:19.003 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.002+0000 7fbcc119c700 1 --2- 172.21.15.40:0/1663213526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcbc0b3010 0x7fbcbc0b0620 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7fbcbc067d50 tx=0x7fbcb0007b50).stop 2020-11-20T13:56:19.004 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.003+0000 7fbcc119c700 1 -- 172.21.15.40:0/1663213526 shutdown_connections 2020-11-20T13:56:19.004 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.003+0000 7fbcc119c700 1 --2- 172.21.15.40:0/1663213526 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fbc9c051290 0x7fbc9c053750 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:19.004 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.003+0000 7fbcc119c700 1 --2- 172.21.15.40:0/1663213526 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbcbc0667b0 0x7fbcbc13e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:19.004 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.003+0000 7fbcc119c700 1 --2- 172.21.15.40:0/1663213526 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbcbc0805f0 0x7fbcbc150790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:19.005 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.003+0000 7fbcc119c700 1 --2- 172.21.15.40:0/1663213526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcbc0b3010 0x7fbcbc0b0620 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:19.005 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.003+0000 7fbcc119c700 1 -- 172.21.15.40:0/1663213526 >> 172.21.15.40:0/1663213526 conn(0x7fbcbc05f390 msgr2=0x7fbcbc0aed00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:19.005 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.003+0000 7fbcc119c700 1 -- 172.21.15.40:0/1663213526 shutdown_connections 2020-11-20T13:56:19.005 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.003+0000 7fbcc119c700 1 -- 172.21.15.40:0/1663213526 wait complete. 2020-11-20T13:56:19.006 INFO:teuthology.orchestra.run.smithi040.stderr:enabled application 'rbd' on pool 'rbd' 2020-11-20T13:56:19.019 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2020-11-20T13:56:19.020 INFO:tasks.ceph.mds.a:Restarting daemon 2020-11-20T13:56:19.020 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-11-20T13:56:19.024 INFO:tasks.ceph.mds.a:Started 2020-11-20T13:56:19.024 INFO:tasks.ceph.mds.b:Restarting daemon 2020-11-20T13:56:19.025 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-11-20T13:56:19.031 INFO:tasks.ceph.mds.b:Started 2020-11-20T13:56:19.031 INFO:tasks.ceph:Setting up CephFS filesystem... 2020-11-20T13:56:19.032 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-20T13:56:19.100 INFO:tasks.ceph.mds.b.smithi116.stdout:starting mds.b at 2020-11-20T13:56:19.113 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2020-11-20T13:56:19.157 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.157+0000 7f68f2048700 1 Processor -- start 2020-11-20T13:56:19.158 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.157+0000 7f68f2048700 1 -- start start 2020-11-20T13:56:19.160 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.158+0000 7f68f2048700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f68ec0b5120 0x7f68ec0b6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:19.160 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.158+0000 7f68f2048700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f68ec12d350 0x7f68ec12d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:19.160 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.158+0000 7f68f2048700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68ec0ba2d0 con 0x7f68ec05c510 2020-11-20T13:56:19.160 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.158+0000 7f68f2048700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68ec12dcb0 con 0x7f68ec0b5120 2020-11-20T13:56:19.161 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.158+0000 7f68f2048700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68ec0ba570 con 0x7f68ec12d350 2020-11-20T13:56:19.161 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.158+0000 7f68eb7fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f68ec05c510 0x7f68ec0baf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44944/0 (socket says 172.21.15.40:44944) 2020-11-20T13:56:19.161 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.158+0000 7f68eb7fe700 1 -- 172.21.15.40:0/3439419018 learned_addr learned my addr 172.21.15.40:0/3439419018 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:19.161 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.158+0000 7f68eaffd700 1 --2- 172.21.15.40:0/3439419018 >> v2:172.21.15.40:3300/0 conn(0x7f68ec0b5120 0x7f68ec0b6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:19.161 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.158+0000 7f68ebfff700 1 --2- 172.21.15.40:0/3439419018 >> v2:172.21.15.116:3300/0 conn(0x7f68ec12d350 0x7f68ec12d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:19.162 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.158+0000 7f68ea7fc700 1 -- 172.21.15.40:0/3439419018 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1802691726 0 0) 0x7f68ec0ba2d0 con 0x7f68ec05c510 2020-11-20T13:56:19.162 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.158+0000 7f68ea7fc700 1 -- 172.21.15.40:0/3439419018 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68cc003230 con 0x7f68ec05c510 2020-11-20T13:56:19.162 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.158+0000 7f68ebfff700 1 -- 172.21.15.40:0/3439419018 >> v2:172.21.15.40:3300/0 conn(0x7f68ec0b5120 msgr2=0x7f68ec0b6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:19.162 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.158+0000 7f68ebfff700 1 --2- 172.21.15.40:0/3439419018 >> v2:172.21.15.40:3300/0 conn(0x7f68ec0b5120 0x7f68ec0b6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:19.162 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.158+0000 7f68ea7fc700 1 -- 172.21.15.40:0/3439419018 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3719884046 0 0) 0x7f68cc003230 con 0x7f68ec05c510 2020-11-20T13:56:19.163 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.158+0000 7f68ebfff700 1 -- 172.21.15.40:0/3439419018 >> v1:172.21.15.40:6789/0 conn(0x7f68ec05c510 legacy=0x7f68ec0baf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:19.163 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.158+0000 7f68ebfff700 1 -- 172.21.15.40:0/3439419018 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68ec0ba810 con 0x7f68ec12d350 2020-11-20T13:56:19.163 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.159+0000 7f68ebfff700 1 --2- 172.21.15.40:0/3439419018 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68ec12d350 0x7f68ec12d7c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f68d4001870 tx=0x7f68e40091a0).ready entity=mon.1 client_cookie=47642818e379b20e server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:19.163 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.159+0000 7f68ea7fc700 1 -- 172.21.15.40:0/3439419018 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68d40016d0 con 0x7f68ec12d350 2020-11-20T13:56:19.163 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.159+0000 7f68ea7fc700 1 -- 172.21.15.40:0/3439419018 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e400b890 con 0x7f68ec12d350 2020-11-20T13:56:19.164 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.159+0000 7f68ea7fc700 1 -- 172.21.15.40:0/3439419018 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68e400ba00 con 0x7f68ec12d350 2020-11-20T13:56:19.164 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.159+0000 7f68f2048700 1 -- 172.21.15.40:0/3439419018 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68ec12d350 msgr2=0x7f68ec12d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:19.164 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.159+0000 7f68f2048700 1 --2- 172.21.15.40:0/3439419018 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68ec12d350 0x7f68ec12d7c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f68d4001870 tx=0x7f68e40091a0).stop 2020-11-20T13:56:19.164 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.159+0000 7f68f2048700 1 -- 172.21.15.40:0/3439419018 shutdown_connections 2020-11-20T13:56:19.164 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.159+0000 7f68f2048700 1 --2- 172.21.15.40:0/3439419018 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68ec12d350 0x7f68ec12d7c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:19.164 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.159+0000 7f68f2048700 1 --2- 172.21.15.40:0/3439419018 >> v2:172.21.15.40:3300/0 conn(0x7f68ec0b5120 0x7f68ec0b6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:19.165 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.159+0000 7f68f2048700 1 -- 172.21.15.40:0/3439419018 >> 172.21.15.40:0/3439419018 conn(0x7f68ec0bdce0 msgr2=0x7f68ec0be0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:19.165 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.159+0000 7f68f2048700 1 -- 172.21.15.40:0/3439419018 shutdown_connections 2020-11-20T13:56:19.165 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.159+0000 7f68f2048700 1 -- 172.21.15.40:0/3439419018 wait complete. 2020-11-20T13:56:19.168 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.160+0000 7f68f2048700 1 Processor -- start 2020-11-20T13:56:19.168 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.160+0000 7f68f2048700 1 -- start start 2020-11-20T13:56:19.168 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.161+0000 7f68f2048700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68ec05c510 0x7f68ec0b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:19.168 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.161+0000 7f68f2048700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68ec0b5120 0x7f68ec12f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:19.168 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.161+0000 7f68f2048700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f68ec12d350 0x7f68ec15a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:19.169 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.161+0000 7f68f2048700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ec0b2470 con 0x7f68ec05c510 2020-11-20T13:56:19.169 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.161+0000 7f68f2048700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ec0b2710 con 0x7f68ec0b5120 2020-11-20T13:56:19.169 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.161+0000 7f68f2048700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68ec0b29b0 con 0x7f68ec12d350 2020-11-20T13:56:19.169 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.161+0000 7f68eb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68ec05c510 0x7f68ec0b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:19.170 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.161+0000 7f68eb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68ec05c510 0x7f68ec0b2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54354/0 (socket says 172.21.15.40:54354) 2020-11-20T13:56:19.170 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.161+0000 7f68eb7fe700 1 -- 172.21.15.40:0/3610221266 learned_addr learned my addr 172.21.15.40:0/3610221266 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:19.170 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.161+0000 7f68eaffd700 1 --2- 172.21.15.40:0/3610221266 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68ec0b5120 0x7f68ec12f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:19.170 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.161+0000 7f68ebfff700 1 --2- 172.21.15.40:0/3610221266 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f68ec12d350 0x7f68ec15a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:19.170 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.162+0000 7f68eb7fe700 1 -- 172.21.15.40:0/3610221266 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f68ec12d350 msgr2=0x7f68ec15a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:19.171 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.162+0000 7f68eb7fe700 1 --2- 172.21.15.40:0/3610221266 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f68ec12d350 0x7f68ec15a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:19.171 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.162+0000 7f68eb7fe700 1 -- 172.21.15.40:0/3610221266 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68ec0b5120 msgr2=0x7f68ec12f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:19.171 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.162+0000 7f68eb7fe700 1 --2- 172.21.15.40:0/3610221266 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68ec0b5120 0x7f68ec12f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:19.171 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.162+0000 7f68eb7fe700 1 -- 172.21.15.40:0/3610221266 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68ec15a9d0 con 0x7f68ec05c510 2020-11-20T13:56:19.172 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.162+0000 7f68eb7fe700 1 --2- 172.21.15.40:0/3610221266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68ec05c510 0x7f68ec0b2e50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f68d4001930 tx=0x7f68d4001160).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:19.172 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.162+0000 7f68e8ff9700 1 -- 172.21.15.40:0/3610221266 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68d4007e10 con 0x7f68ec05c510 2020-11-20T13:56:19.172 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.162+0000 7f68e8ff9700 1 -- 172.21.15.40:0/3610221266 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d4007170 con 0x7f68ec05c510 2020-11-20T13:56:19.172 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.162+0000 7f68f2048700 1 -- 172.21.15.40:0/3610221266 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68ec0b20f0 con 0x7f68ec05c510 2020-11-20T13:56:19.172 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.162+0000 7f68e8ff9700 1 -- 172.21.15.40:0/3610221266 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68d400a7f0 con 0x7f68ec05c510 2020-11-20T13:56:19.173 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.162+0000 7f68f2048700 1 -- 172.21.15.40:0/3610221266 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68ec0af510 con 0x7f68ec05c510 2020-11-20T13:56:19.173 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.163+0000 7f68f2048700 1 -- 172.21.15.40:0/3610221266 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68ec0b2b70 con 0x7f68ec05c510 2020-11-20T13:56:19.173 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.163+0000 7f68e8ff9700 1 -- 172.21.15.40:0/3610221266 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f68d4010070 con 0x7f68ec05c510 2020-11-20T13:56:19.173 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.163+0000 7f68e8ff9700 1 --2- 172.21.15.40:0/3610221266 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f68d0051290 0x7f68d0053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:19.173 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.163+0000 7f68e8ff9700 1 -- 172.21.15.40:0/3610221266 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f68d4068730 con 0x7f68ec05c510 2020-11-20T13:56:19.174 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.163+0000 7f68eaffd700 1 --2- 172.21.15.40:0/3610221266 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f68d0051290 0x7f68d0053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:19.174 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.164+0000 7f68eaffd700 1 --2- 172.21.15.40:0/3610221266 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f68d0051290 0x7f68d0053750 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f68dc00b000 tx=0x7f68dc006dc0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:19.174 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.167+0000 7f68e8ff9700 1 -- 172.21.15.40:0/3610221266 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f68d4040de0 con 0x7f68ec05c510 2020-11-20T13:56:19.329 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.328+0000 7f68f2048700 1 -- 172.21.15.40:0/3610221266 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68ec057050 con 0x7f68ec05c510 2020-11-20T13:56:19.330 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.329+0000 7f68e8ff9700 1 -- 172.21.15.40:0/3610221266 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+13239 (secure 0 0 0) 0x7f68d4044400 con 0x7f68ec05c510 2020-11-20T13:56:19.330 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:56:19.331 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":17,"fsid":"da1ed737-5174-430f-a3c9-56f6bbfad13c","created":"2020-11-20T13:56:04.990651+0000","modified":"2020-11-20T13:56:18.991513+0000","last_up_change":"2020-11-20T13:56:13.893026+0000","last_in_change":"2020-11-20T13:56:13.893026+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T13:56:08.330017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T13:56:17.939684+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"9ea4c667-8938-46d6-a5ca-092d55c1f79e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":22825},{"type":"v1","addr":"172.21.15.40:6818","nonce":22825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":22825},{"type":"v1","addr":"172.21.15.40:6822","nonce":22825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":22825},{"type":"v1","addr":"172.21.15.40:6830","nonce":22825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":22825},{"type":"v1","addr":"172.21.15.40:6826","nonce":22825}]},"public_addr":"172.21.15.40:6818/22825","cluster_addr":"172.21.15.40:6822/22825","heartbeat_back_addr":"172.21.15.40:6830/22825","heartbeat_front_addr":"172.21.15.40:6826/22825","state":["exists","up"]},{"osd":1,"uuid":"515708d9-5087-4e55-8d14-cdea1c00eacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":22823},{"type":"v1","addr":"172.21.15.40:6801","nonce":22823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":22823},{"type":"v1","addr":"172.21.15.40:6803","nonce":22823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":22823},{"type":"v1","addr":"172.21.15.40:6807","nonce":22823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":22823},{"type":"v1","addr":"172.21.15.40:6805","nonce":22823}]},"public_addr":"172.21.15.40:6801/22823","cluster_addr":"172.21.15.40:6803/22823","heartbeat_back_addr":"172.21.15.40:6807/22823","heartbeat_front_addr":"172.21.15.40:6805/22823","state":["exists","up"]},{"osd":2,"uuid":"4d86bca7-743e-4983-b35f-36676c405b36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":22824},{"type":"v1","addr":"172.21.15.40:6809","nonce":22824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":22824},{"type":"v1","addr":"172.21.15.40:6811","nonce":22824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":22824},{"type":"v1","addr":"172.21.15.40:6815","nonce":22824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":22824},{"type":"v1","addr":"172.21.15.40:6813","nonce":22824}]},"public_addr":"172.21.15.40:6809/22824","cluster_addr":"172.21.15.40:6811/22824","heartbeat_back_addr":"172.21.15.40:6815/22824","heartbeat_front_addr":"172.21.15.40:6813/22824","state":["exists","up"]},{"osd":3,"uuid":"f4dc726b-6b34-4e8b-8eeb-87060b58002d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":22826},{"type":"v1","addr":"172.21.15.40:6819","nonce":22826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":22826},{"type":"v1","addr":"172.21.15.40:6823","nonce":22826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":22826},{"type":"v1","addr":"172.21.15.40:6831","nonce":22826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":22826},{"type":"v1","addr":"172.21.15.40:6827","nonce":22826}]},"public_addr":"172.21.15.40:6819/22826","cluster_addr":"172.21.15.40:6823/22826","heartbeat_back_addr":"172.21.15.40:6831/22826","heartbeat_front_addr":"172.21.15.40:6827/22826","state":["exists","up"]},{"osd":4,"uuid":"25a6d53b-d1f0-4498-b7d5-ffa38661d403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":23181},{"type":"v1","addr":"172.21.15.116:6819","nonce":23181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":23181},{"type":"v1","addr":"172.21.15.116:6821","nonce":23181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":23181},{"type":"v1","addr":"172.21.15.116:6825","nonce":23181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":23181},{"type":"v1","addr":"172.21.15.116:6823","nonce":23181}]},"public_addr":"172.21.15.116:6819/23181","cluster_addr":"172.21.15.116:6821/23181","heartbeat_back_addr":"172.21.15.116:6825/23181","heartbeat_front_addr":"172.21.15.116:6823/23181","state":["exists","up"]},{"osd":5,"uuid":"7d7a5af8-229b-4351-9a21-a3d509cd4635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":23171},{"type":"v1","addr":"172.21.15.116:6803","nonce":23171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":23171},{"type":"v1","addr":"172.21.15.116:6805","nonce":23171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":23171},{"type":"v1","addr":"172.21.15.116:6809","nonce":23171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":23171},{"type":"v1","addr":"172.21.15.116:6807","nonce":23171}]},"public_addr":"172.21.15.116:6803/23171","cluster_addr":"172.21.15.116:6805/23171","heartbeat_back_addr":"172.21.15.116:6809/23171","heartbeat_front_addr":"172.21.15.116:6807/23171","state":["exists","up"]},{"osd":6,"uuid":"8247d248-a140-4e36-9796-9847a97df58d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":23172},{"type":"v1","addr":"172.21.15.116:6811","nonce":23172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":23172},{"type":"v1","addr":"172.21.15.116:6813","nonce":23172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":23172},{"type":"v1","addr":"172.21.15.116:6817","nonce":23172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":23172},{"type":"v1","addr":"172.21.15.116:6815","nonce":23172}]},"public_addr":"172.21.15.116:6811/23172","cluster_addr":"172.21.15.116:6813/23172","heartbeat_back_addr":"172.21.15.116:6817/23172","heartbeat_front_addr":"172.21.15.116:6815/23172","state":["exists","up"]},{"osd":7,"uuid":"472803c4-70b8-499d-b18a-71aaeec6aab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":23196},{"type":"v1","addr":"172.21.15.116:6827","nonce":23196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":23196},{"type":"v1","addr":"172.21.15.116:6829","nonce":23196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":23196},{"type":"v1","addr":"172.21.15.116:6833","nonce":23196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":23196},{"type":"v1","addr":"172.21.15.116:6831","nonce":23196}]},"public_addr":"172.21.15.116:6827/23196","cluster_addr":"172.21.15.116:6829/23196","heartbeat_back_addr":"172.21.15.116:6833/23196","heartbeat_front_addr":"172.21.15.116:6831/23196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.029357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:11.989797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.091097+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.014766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.070860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.071864+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.063400+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.109028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-20T13:56:19.334 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.330+0000 7f68f2048700 1 -- 172.21.15.40:0/3610221266 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f68d0051290 msgr2=0x7f68d0053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:19.334 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.330+0000 7f68f2048700 1 --2- 172.21.15.40:0/3610221266 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f68d0051290 0x7f68d0053750 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f68dc00b000 tx=0x7f68dc006dc0).stop 2020-11-20T13:56:19.335 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.331+0000 7f68f2048700 1 -- 172.21.15.40:0/3610221266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68ec05c510 msgr2=0x7f68ec0b2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:19.335 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.331+0000 7f68f2048700 1 --2- 172.21.15.40:0/3610221266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68ec05c510 0x7f68ec0b2e50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f68d4001930 tx=0x7f68d4001160).stop 2020-11-20T13:56:19.335 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.331+0000 7f68f2048700 1 -- 172.21.15.40:0/3610221266 shutdown_connections 2020-11-20T13:56:19.336 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.331+0000 7f68f2048700 1 --2- 172.21.15.40:0/3610221266 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f68d0051290 0x7f68d0053750 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:19.336 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.331+0000 7f68f2048700 1 --2- 172.21.15.40:0/3610221266 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f68ec12d350 0x7f68ec15a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:19.336 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.331+0000 7f68f2048700 1 --2- 172.21.15.40:0/3610221266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68ec05c510 0x7f68ec0b2e50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:19.336 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.331+0000 7f68f2048700 1 --2- 172.21.15.40:0/3610221266 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68ec0b5120 0x7f68ec12f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:19.336 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.331+0000 7f68f2048700 1 -- 172.21.15.40:0/3610221266 >> 172.21.15.40:0/3610221266 conn(0x7f68ec0bdce0 msgr2=0x7f68ec148700 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:19.337 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.331+0000 7f68f2048700 1 -- 172.21.15.40:0/3610221266 shutdown_connections 2020-11-20T13:56:19.337 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.331+0000 7f68f2048700 1 -- 172.21.15.40:0/3610221266 wait complete. 2020-11-20T13:56:19.341 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T13:56:08.330017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T13:56:17.939684+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-11-20T13:56:19.342 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-20T13:56:19.459 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.459+0000 7fa85323e700 1 Processor -- start 2020-11-20T13:56:19.460 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.459+0000 7fa85323e700 1 -- start start 2020-11-20T13:56:19.462 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.460+0000 7fa85323e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa84c0b6c80 0x7fa84c0b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:19.462 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.460+0000 7fa85323e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa84c144bd0 0x7fa84c0b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:19.462 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.460+0000 7fa85323e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa84c0b0d10 con 0x7fa84c0b45e0 2020-11-20T13:56:19.462 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.460+0000 7fa85323e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa84c145060 con 0x7fa84c0b6c80 2020-11-20T13:56:19.463 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.460+0000 7fa85323e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa84c0b0fb0 con 0x7fa84c144bd0 2020-11-20T13:56:19.463 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.460+0000 7fa850fda700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa84c0b6c80 0x7fa84c0b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:19.463 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.460+0000 7fa850fda700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa84c0b6c80 0x7fa84c0b31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54362/0 (socket says 172.21.15.40:54362) 2020-11-20T13:56:19.463 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.460+0000 7fa850fda700 1 -- 172.21.15.40:0/4131494670 learned_addr learned my addr 172.21.15.40:0/4131494670 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:19.463 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.460+0000 7fa84bfff700 1 --2- 172.21.15.40:0/4131494670 >> v2:172.21.15.116:3300/0 conn(0x7fa84c144bd0 0x7fa84c0b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:19.464 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.460+0000 7fa850fda700 1 -- 172.21.15.40:0/4131494670 >> v2:172.21.15.116:3300/0 conn(0x7fa84c144bd0 msgr2=0x7fa84c0b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:19.464 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.460+0000 7fa850fda700 1 --2- 172.21.15.40:0/4131494670 >> v2:172.21.15.116:3300/0 conn(0x7fa84c144bd0 0x7fa84c0b40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:19.464 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.460+0000 7fa850fda700 1 -- 172.21.15.40:0/4131494670 >> v1:172.21.15.116:6789/0 conn(0x7fa84c0b45e0 legacy=0x7fa84c0b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:19.464 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.460+0000 7fa850fda700 1 -- 172.21.15.40:0/4131494670 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa84c0b7060 con 0x7fa84c0b6c80 2020-11-20T13:56:19.464 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.460+0000 7fa850fda700 1 --2- 172.21.15.40:0/4131494670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa84c0b6c80 0x7fa84c0b31f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fa83c002f30 tx=0x7fa83c006e20).ready entity=mon.0 client_cookie=21a562785b32847f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:19.465 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.461+0000 7fa84b7fe700 1 -- 172.21.15.40:0/4131494670 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa83c009960 con 0x7fa84c0b6c80 2020-11-20T13:56:19.465 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.461+0000 7fa84b7fe700 1 -- 172.21.15.40:0/4131494670 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa83c009ac0 con 0x7fa84c0b6c80 2020-11-20T13:56:19.465 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.461+0000 7fa84b7fe700 1 -- 172.21.15.40:0/4131494670 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa83c009e20 con 0x7fa84c0b6c80 2020-11-20T13:56:19.465 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.461+0000 7fa85323e700 1 -- 172.21.15.40:0/4131494670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa84c0b6c80 msgr2=0x7fa84c0b31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:19.466 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.461+0000 7fa85323e700 1 --2- 172.21.15.40:0/4131494670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa84c0b6c80 0x7fa84c0b31f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fa83c002f30 tx=0x7fa83c006e20).stop 2020-11-20T13:56:19.466 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.461+0000 7fa85323e700 1 -- 172.21.15.40:0/4131494670 shutdown_connections 2020-11-20T13:56:19.466 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.461+0000 7fa85323e700 1 --2- 172.21.15.40:0/4131494670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa84c0b6c80 0x7fa84c0b31f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:19.466 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.461+0000 7fa85323e700 1 --2- 172.21.15.40:0/4131494670 >> v2:172.21.15.116:3300/0 conn(0x7fa84c144bd0 0x7fa84c0b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:19.467 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.461+0000 7fa85323e700 1 -- 172.21.15.40:0/4131494670 >> 172.21.15.40:0/4131494670 conn(0x7fa84c0ba1d0 msgr2=0x7fa84c0b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:19.467 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.461+0000 7fa85323e700 1 -- 172.21.15.40:0/4131494670 shutdown_connections 2020-11-20T13:56:19.467 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.461+0000 7fa85323e700 1 -- 172.21.15.40:0/4131494670 wait complete. 2020-11-20T13:56:19.470 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.462+0000 7fa85323e700 1 Processor -- start 2020-11-20T13:56:19.470 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.462+0000 7fa85323e700 1 -- start start 2020-11-20T13:56:19.471 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.463+0000 7fa85323e700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa84c0b45e0 0x7fa84c0672c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:19.471 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.463+0000 7fa85323e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa84c0b6c80 0x7fa84c065c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:19.471 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.463+0000 7fa85323e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa84c144bd0 0x7fa84c12d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:19.471 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.463+0000 7fa85323e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c12d670 con 0x7fa84c144bd0 2020-11-20T13:56:19.472 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.463+0000 7fa85323e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c157800 con 0x7fa84c0b6c80 2020-11-20T13:56:19.472 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.463+0000 7fa85323e700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c157aa0 con 0x7fa84c0b45e0 2020-11-20T13:56:19.472 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.463+0000 7fa850fda700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa84c0b45e0 0x7fa84c0672c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:19.473 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.463+0000 7fa8517db700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa84c144bd0 0x7fa84c12d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:19.473 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.463+0000 7fa84bfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa84c0b6c80 0x7fa84c065c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:19.473 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.463+0000 7fa8517db700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa84c144bd0 0x7fa84c12d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54372/0 (socket says 172.21.15.40:54372) 2020-11-20T13:56:19.473 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.463+0000 7fa8517db700 1 -- 172.21.15.40:0/4258836368 learned_addr learned my addr 172.21.15.40:0/4258836368 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:19.474 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.463+0000 7fa850fda700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa84c0b45e0 0x7fa84c0672c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.40:39442/0 (socket says 172.21.15.40:39442) 2020-11-20T13:56:19.474 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.463+0000 7fa8517db700 1 -- 172.21.15.40:0/4258836368 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa84c0b45e0 msgr2=0x7fa84c0672c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:19.474 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.463+0000 7fa8517db700 1 --2- 172.21.15.40:0/4258836368 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa84c0b45e0 0x7fa84c0672c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:19.474 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.463+0000 7fa8517db700 1 -- 172.21.15.40:0/4258836368 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa84c0b6c80 msgr2=0x7fa84c065c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:19.474 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.463+0000 7fa8517db700 1 --2- 172.21.15.40:0/4258836368 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa84c0b6c80 0x7fa84c065c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:19.475 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.463+0000 7fa8517db700 1 -- 172.21.15.40:0/4258836368 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa84c157d40 con 0x7fa84c144bd0 2020-11-20T13:56:19.475 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.464+0000 7fa8517db700 1 --2- 172.21.15.40:0/4258836368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa84c144bd0 0x7fa84c12d180 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fa84c0687d0 tx=0x7fa844007e80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:19.475 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.464+0000 7fa849ffb700 1 -- 172.21.15.40:0/4258836368 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa84400a5a0 con 0x7fa84c144bd0 2020-11-20T13:56:19.475 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.464+0000 7fa849ffb700 1 -- 172.21.15.40:0/4258836368 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa84400a700 con 0x7fa84c144bd0 2020-11-20T13:56:19.476 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.464+0000 7fa85323e700 1 -- 172.21.15.40:0/4258836368 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa84c157fe0 con 0x7fa84c144bd0 2020-11-20T13:56:19.476 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.464+0000 7fa849ffb700 1 -- 172.21.15.40:0/4258836368 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa84400aa60 con 0x7fa84c144bd0 2020-11-20T13:56:19.476 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.464+0000 7fa85323e700 1 -- 172.21.15.40:0/4258836368 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa84c158750 con 0x7fa84c144bd0 2020-11-20T13:56:19.476 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.465+0000 7fa85323e700 1 -- 172.21.15.40:0/4258836368 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa84c057050 con 0x7fa84c144bd0 2020-11-20T13:56:19.477 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.465+0000 7fa849ffb700 1 -- 172.21.15.40:0/4258836368 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fa844019030 con 0x7fa84c144bd0 2020-11-20T13:56:19.477 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.465+0000 7fa849ffb700 1 --2- 172.21.15.40:0/4258836368 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa82c051320 0x7fa82c0537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:19.477 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.465+0000 7fa849ffb700 1 -- 172.21.15.40:0/4258836368 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fa8440684d0 con 0x7fa84c144bd0 2020-11-20T13:56:19.477 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.465+0000 7fa850fda700 1 --2- 172.21.15.40:0/4258836368 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa82c051320 0x7fa82c0537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:19.478 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.466+0000 7fa850fda700 1 --2- 172.21.15.40:0/4258836368 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa82c051320 0x7fa82c0537e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fa83c01b000 tx=0x7fa83c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:19.478 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.469+0000 7fa849ffb700 1 -- 172.21.15.40:0/4258836368 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fa844040b80 con 0x7fa84c144bd0 2020-11-20T13:56:19.628 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.628+0000 7fa85323e700 1 -- 172.21.15.40:0/4258836368 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fa84c158170 con 0x7fa84c144bd0 2020-11-20T13:56:19.629 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.628+0000 7fa849ffb700 1 -- 172.21.15.40:0/4258836368 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v17) v1 ==== 110+0+10 (secure 0 0 0) 0x7fa8440441a0 con 0x7fa84c144bd0 2020-11-20T13:56:19.629 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2020-11-20T13:56:19.630 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.630+0000 7fa85323e700 1 -- 172.21.15.40:0/4258836368 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa82c051320 msgr2=0x7fa82c0537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:19.630 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.630+0000 7fa85323e700 1 --2- 172.21.15.40:0/4258836368 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa82c051320 0x7fa82c0537e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fa83c01b000 tx=0x7fa83c015000).stop 2020-11-20T13:56:19.631 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.630+0000 7fa85323e700 1 -- 172.21.15.40:0/4258836368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa84c144bd0 msgr2=0x7fa84c12d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:19.631 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.630+0000 7fa85323e700 1 --2- 172.21.15.40:0/4258836368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa84c144bd0 0x7fa84c12d180 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fa84c0687d0 tx=0x7fa844007e80).stop 2020-11-20T13:56:19.631 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.630+0000 7fa85323e700 1 -- 172.21.15.40:0/4258836368 shutdown_connections 2020-11-20T13:56:19.632 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.630+0000 7fa85323e700 1 --2- 172.21.15.40:0/4258836368 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa82c051320 0x7fa82c0537e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:19.632 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.630+0000 7fa85323e700 1 --2- 172.21.15.40:0/4258836368 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa84c0b45e0 0x7fa84c0672c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:19.633 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.630+0000 7fa85323e700 1 --2- 172.21.15.40:0/4258836368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa84c144bd0 0x7fa84c12d180 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:19.633 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.630+0000 7fa85323e700 1 --2- 172.21.15.40:0/4258836368 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa84c0b6c80 0x7fa84c065c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:19.633 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.630+0000 7fa85323e700 1 -- 172.21.15.40:0/4258836368 >> 172.21.15.40:0/4258836368 conn(0x7fa84c0ba1d0 msgr2=0x7fa84c12b620 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:19.633 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.630+0000 7fa85323e700 1 -- 172.21.15.40:0/4258836368 shutdown_connections 2020-11-20T13:56:19.634 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.630+0000 7fa85323e700 1 -- 172.21.15.40:0/4258836368 wait complete. 2020-11-20T13:56:19.640 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-20T13:56:19.757 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.757+0000 7f9844cb9700 1 Processor -- start 2020-11-20T13:56:19.757 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.757+0000 7f9844cb9700 1 -- start start 2020-11-20T13:56:19.759 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.757+0000 7f9844cb9700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f98400b5bf0 0x7f984005f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:19.760 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.758+0000 7f9844cb9700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98400b5fd0 con 0x7f98400b3960 2020-11-20T13:56:19.760 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.758+0000 7f9844cb9700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98400b05f0 con 0x7f9840144390 2020-11-20T13:56:19.760 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.758+0000 7f9844cb9700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9840144870 con 0x7f98400b5bf0 2020-11-20T13:56:19.762 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.758+0000 7f983ed9d700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f98400b3960 0x7f984014cf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44974/0 (socket says 172.21.15.40:44974) 2020-11-20T13:56:19.762 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.758+0000 7f983ed9d700 1 -- 172.21.15.40:0/644458009 learned_addr learned my addr 172.21.15.40:0/644458009 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:19.762 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.758+0000 7f983e59c700 1 --2- 172.21.15.40:0/644458009 >> v2:172.21.15.116:3300/0 conn(0x7f98400b5bf0 0x7f984005f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:19.762 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.758+0000 7f983d59a700 1 -- 172.21.15.40:0/644458009 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 715636973 0 0) 0x7f98400b5fd0 con 0x7f98400b3960 2020-11-20T13:56:19.763 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.758+0000 7f983d59a700 1 -- 172.21.15.40:0/644458009 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9820004340 con 0x7f98400b3960 2020-11-20T13:56:19.763 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.758+0000 7f983e59c700 1 -- 172.21.15.40:0/644458009 >> v1:172.21.15.116:6789/0 conn(0x7f9840144390 legacy=0x7f984005f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:19.763 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.758+0000 7f983e59c700 1 -- 172.21.15.40:0/644458009 >> v1:172.21.15.40:6789/0 conn(0x7f98400b3960 legacy=0x7f984014cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:19.763 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.758+0000 7f983d59a700 1 -- 172.21.15.40:0/644458009 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1729116806 0 0) 0x7f98400b05f0 con 0x7f9840144390 2020-11-20T13:56:19.763 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.758+0000 7f983e59c700 1 -- 172.21.15.40:0/644458009 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98400b0c30 con 0x7f98400b5bf0 2020-11-20T13:56:19.764 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.758+0000 7f983e59c700 1 --2- 172.21.15.40:0/644458009 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98400b5bf0 0x7f984005f390 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f9828002f30 tx=0x7f9828006d60).ready entity=mon.1 client_cookie=993118e067f0885 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:19.764 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.758+0000 7f983d59a700 1 -- 172.21.15.40:0/644458009 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9828009600 con 0x7f98400b5bf0 2020-11-20T13:56:19.764 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.758+0000 7f983d59a700 1 -- 172.21.15.40:0/644458009 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9828009760 con 0x7f98400b5bf0 2020-11-20T13:56:19.764 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.758+0000 7f983d59a700 1 -- 172.21.15.40:0/644458009 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98280098d0 con 0x7f98400b5bf0 2020-11-20T13:56:19.764 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.758+0000 7f9844cb9700 1 -- 172.21.15.40:0/644458009 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98400b5bf0 msgr2=0x7f984005f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:19.767 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.758+0000 7f9844cb9700 1 --2- 172.21.15.40:0/644458009 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98400b5bf0 0x7f984005f390 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f9828002f30 tx=0x7f9828006d60).stop 2020-11-20T13:56:19.767 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.759+0000 7f9844cb9700 1 -- 172.21.15.40:0/644458009 shutdown_connections 2020-11-20T13:56:19.768 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.759+0000 7f9844cb9700 1 --2- 172.21.15.40:0/644458009 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98400b5bf0 0x7f984005f390 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:19.768 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.759+0000 7f9844cb9700 1 -- 172.21.15.40:0/644458009 >> 172.21.15.40:0/644458009 conn(0x7f98400ba1d0 msgr2=0x7f98400b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:19.768 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.759+0000 7f9844cb9700 1 -- 172.21.15.40:0/644458009 shutdown_connections 2020-11-20T13:56:19.768 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.759+0000 7f9844cb9700 1 -- 172.21.15.40:0/644458009 wait complete. 2020-11-20T13:56:19.771 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.760+0000 7f9844cb9700 1 Processor -- start 2020-11-20T13:56:19.771 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.760+0000 7f9844cb9700 1 -- start start 2020-11-20T13:56:19.771 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.761+0000 7f9844cb9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98400b3960 0x7f98400aee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:19.772 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.761+0000 7f9844cb9700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98400b5bf0 0x7f98401559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:19.772 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.761+0000 7f9844cb9700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9840144390 0x7f9840152460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:19.772 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.761+0000 7f9844cb9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98400631a0 con 0x7f98400b3960 2020-11-20T13:56:19.772 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.761+0000 7f9844cb9700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9840155ed0 con 0x7f98400b5bf0 2020-11-20T13:56:19.772 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.761+0000 7f9844cb9700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9840151d20 con 0x7f9840144390 2020-11-20T13:56:19.773 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.761+0000 7f983e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98400b3960 0x7f98400aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:19.773 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.761+0000 7f983e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98400b3960 0x7f98400aee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54380/0 (socket says 172.21.15.40:54380) 2020-11-20T13:56:19.773 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.761+0000 7f983e59c700 1 -- 172.21.15.40:0/2943007225 learned_addr learned my addr 172.21.15.40:0/2943007225 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:19.773 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.761+0000 7f983ed9d700 1 --2- 172.21.15.40:0/2943007225 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9840144390 0x7f9840152460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:19.774 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.761+0000 7f983dd9b700 1 --2- 172.21.15.40:0/2943007225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98400b5bf0 0x7f98401559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:19.774 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.761+0000 7f983e59c700 1 -- 172.21.15.40:0/2943007225 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9840144390 msgr2=0x7f9840152460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:19.774 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.761+0000 7f983e59c700 1 --2- 172.21.15.40:0/2943007225 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9840144390 0x7f9840152460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:19.774 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.761+0000 7f983e59c700 1 -- 172.21.15.40:0/2943007225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98400b5bf0 msgr2=0x7f98401559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:19.775 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.761+0000 7f983e59c700 1 --2- 172.21.15.40:0/2943007225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98400b5bf0 0x7f98401559e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:19.775 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.761+0000 7f983e59c700 1 -- 172.21.15.40:0/2943007225 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9840151fc0 con 0x7f98400b3960 2020-11-20T13:56:19.776 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.761+0000 7f983e59c700 1 --2- 172.21.15.40:0/2943007225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98400b3960 0x7f98400aee00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f9840156060 tx=0x7f9828016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:19.776 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.761+0000 7f98377fe700 1 -- 172.21.15.40:0/2943007225 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98280097f0 con 0x7f98400b3960 2020-11-20T13:56:19.777 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.761+0000 7f98377fe700 1 -- 172.21.15.40:0/2943007225 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9828009950 con 0x7f98400b3960 2020-11-20T13:56:19.777 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.761+0000 7f9844cb9700 1 -- 172.21.15.40:0/2943007225 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9840152950 con 0x7f98400b3960 2020-11-20T13:56:19.777 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.761+0000 7f98377fe700 1 -- 172.21.15.40:0/2943007225 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9828009cb0 con 0x7f98400b3960 2020-11-20T13:56:19.777 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.761+0000 7f9844cb9700 1 -- 172.21.15.40:0/2943007225 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9840062aa0 con 0x7f98400b3960 2020-11-20T13:56:19.778 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.763+0000 7f98377fe700 1 -- 172.21.15.40:0/2943007225 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f982800f070 con 0x7f98400b3960 2020-11-20T13:56:19.778 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.763+0000 7f9844cb9700 1 -- 172.21.15.40:0/2943007225 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9840152150 con 0x7f98400b3960 2020-11-20T13:56:19.778 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.763+0000 7f98377fe700 1 --2- 172.21.15.40:0/2943007225 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f98240513c0 0x7f9824053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:19.778 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.763+0000 7f98377fe700 1 -- 172.21.15.40:0/2943007225 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f9828069130 con 0x7f98400b3960 2020-11-20T13:56:19.779 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.763+0000 7f983dd9b700 1 --2- 172.21.15.40:0/2943007225 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f98240513c0 0x7f9824053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:19.779 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.763+0000 7f983dd9b700 1 --2- 172.21.15.40:0/2943007225 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f98240513c0 0x7f9824053880 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f983000b000 tx=0x7f9830009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:19.779 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.767+0000 7f98377fe700 1 -- 172.21.15.40:0/2943007225 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f98280418a0 con 0x7f98400b3960 2020-11-20T13:56:19.929 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.928+0000 7f9844cb9700 1 -- 172.21.15.40:0/2943007225 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9840057050 con 0x7f98400b3960 2020-11-20T13:56:19.929 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.929+0000 7f98377fe700 1 -- 172.21.15.40:0/2943007225 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7f98280240a0 con 0x7f98400b3960 2020-11-20T13:56:19.929 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2020-11-20T13:56:19.930 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.930+0000 7f9844cb9700 1 -- 172.21.15.40:0/2943007225 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f98240513c0 msgr2=0x7f9824053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:19.931 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.930+0000 7f9844cb9700 1 --2- 172.21.15.40:0/2943007225 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f98240513c0 0x7f9824053880 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f983000b000 tx=0x7f9830009040).stop 2020-11-20T13:56:19.931 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.930+0000 7f9844cb9700 1 -- 172.21.15.40:0/2943007225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98400b3960 msgr2=0x7f98400aee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:19.931 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.930+0000 7f9844cb9700 1 --2- 172.21.15.40:0/2943007225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98400b3960 0x7f98400aee00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f9840156060 tx=0x7f9828016000).stop 2020-11-20T13:56:19.931 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.930+0000 7f9844cb9700 1 -- 172.21.15.40:0/2943007225 shutdown_connections 2020-11-20T13:56:19.932 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.930+0000 7f9844cb9700 1 --2- 172.21.15.40:0/2943007225 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f98240513c0 0x7f9824053880 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:19.932 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.930+0000 7f9844cb9700 1 --2- 172.21.15.40:0/2943007225 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9840144390 0x7f9840152460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:19.932 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.930+0000 7f9844cb9700 1 --2- 172.21.15.40:0/2943007225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98400b3960 0x7f98400aee00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:19.932 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.930+0000 7f9844cb9700 1 --2- 172.21.15.40:0/2943007225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98400b5bf0 0x7f98401559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:19.932 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.930+0000 7f9844cb9700 1 -- 172.21.15.40:0/2943007225 >> 172.21.15.40:0/2943007225 conn(0x7f98400ba1d0 msgr2=0x7f98400b4160 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:19.933 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.930+0000 7f9844cb9700 1 -- 172.21.15.40:0/2943007225 shutdown_connections 2020-11-20T13:56:19.933 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:19.930+0000 7f9844cb9700 1 -- 172.21.15.40:0/2943007225 wait complete. 2020-11-20T13:56:19.941 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2020-11-20T13:56:20.056 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.056+0000 7f2312f04700 1 Processor -- start 2020-11-20T13:56:20.056 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.056+0000 7f2312f04700 1 -- start start 2020-11-20T13:56:20.058 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.056+0000 7f2312f04700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f230c09bc10 0x7f230c0aff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:20.060 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.056+0000 7f2312f04700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f230c0b81c0 0x7f230c0b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:20.061 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.057+0000 7f2312f04700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f230c09bff0 con 0x7f230c099f10 2020-11-20T13:56:20.061 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.057+0000 7f2312f04700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f230c0b8740 con 0x7f230c09bc10 2020-11-20T13:56:20.061 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.057+0000 7f2312f04700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f230c0b89e0 con 0x7f230c0b81c0 2020-11-20T13:56:20.061 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.057+0000 7f2310ca0700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f230c09bc10 0x7f230c0aff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:20.062 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.057+0000 7f2310ca0700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f230c09bc10 0x7f230c0aff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54392/0 (socket says 172.21.15.40:54392) 2020-11-20T13:56:20.062 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.057+0000 7f2310ca0700 1 -- 172.21.15.40:0/2732546218 learned_addr learned my addr 172.21.15.40:0/2732546218 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:20.062 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.057+0000 7f2303fff700 1 --2- 172.21.15.40:0/2732546218 >> v2:172.21.15.116:3300/0 conn(0x7f230c0b81c0 0x7f230c0b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:20.062 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.057+0000 7f2310ca0700 1 -- 172.21.15.40:0/2732546218 >> v2:172.21.15.116:3300/0 conn(0x7f230c0b81c0 msgr2=0x7f230c0b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:20.062 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.057+0000 7f2310ca0700 1 --2- 172.21.15.40:0/2732546218 >> v2:172.21.15.116:3300/0 conn(0x7f230c0b81c0 0x7f230c0b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:20.063 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.057+0000 7f2310ca0700 1 -- 172.21.15.40:0/2732546218 >> v1:172.21.15.116:6789/0 conn(0x7f230c099f10 legacy=0x7f230c09a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:20.063 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.057+0000 7f2310ca0700 1 -- 172.21.15.40:0/2732546218 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f230c0b8c80 con 0x7f230c09bc10 2020-11-20T13:56:20.063 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.057+0000 7f2303fff700 1 --2- 172.21.15.40:0/2732546218 >> v2:172.21.15.116:3300/0 conn(0x7f230c0b81c0 0x7f230c0b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:20.063 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.057+0000 7f2310ca0700 1 --2- 172.21.15.40:0/2732546218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f230c09bc10 0x7f230c0aff40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f230c0627d0 tx=0x7f2308006e80).ready entity=mon.0 client_cookie=f7b7a35013aaf55b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:20.063 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.057+0000 7f23037fe700 1 -- 172.21.15.40:0/2732546218 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23080097b0 con 0x7f230c09bc10 2020-11-20T13:56:20.064 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.057+0000 7f23037fe700 1 -- 172.21.15.40:0/2732546218 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2308009910 con 0x7f230c09bc10 2020-11-20T13:56:20.064 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.057+0000 7f23037fe700 1 -- 172.21.15.40:0/2732546218 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2308009a80 con 0x7f230c09bc10 2020-11-20T13:56:20.064 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.057+0000 7f2312f04700 1 -- 172.21.15.40:0/2732546218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f230c09bc10 msgr2=0x7f230c0aff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:20.065 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.057+0000 7f2312f04700 1 --2- 172.21.15.40:0/2732546218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f230c09bc10 0x7f230c0aff40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f230c0627d0 tx=0x7f2308006e80).stop 2020-11-20T13:56:20.066 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.058+0000 7f2312f04700 1 -- 172.21.15.40:0/2732546218 shutdown_connections 2020-11-20T13:56:20.066 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.058+0000 7f2312f04700 1 --2- 172.21.15.40:0/2732546218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f230c09bc10 0x7f230c0aff40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:20.066 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.058+0000 7f2312f04700 1 --2- 172.21.15.40:0/2732546218 >> v2:172.21.15.116:3300/0 conn(0x7f230c0b81c0 0x7f230c0b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:20.066 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.058+0000 7f2312f04700 1 -- 172.21.15.40:0/2732546218 >> 172.21.15.40:0/2732546218 conn(0x7f230c0ba380 msgr2=0x7f230c0b6350 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:20.067 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.058+0000 7f2312f04700 1 -- 172.21.15.40:0/2732546218 shutdown_connections 2020-11-20T13:56:20.067 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.058+0000 7f2312f04700 1 -- 172.21.15.40:0/2732546218 wait complete. 2020-11-20T13:56:20.070 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.059+0000 7f2312f04700 1 Processor -- start 2020-11-20T13:56:20.071 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.059+0000 7f2312f04700 1 -- start start 2020-11-20T13:56:20.071 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.060+0000 7f2312f04700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f230c099f10 0x7f230c067640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:20.071 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.060+0000 7f2312f04700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f230c09bc10 0x7f230c1456e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:20.071 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.060+0000 7f2312f04700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f230c0b81c0 0x7f230c1567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:20.072 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.060+0000 7f2312f04700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230c13b6c0 con 0x7f230c099f10 2020-11-20T13:56:20.072 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.060+0000 7f2312f04700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230c142910 con 0x7f230c09bc10 2020-11-20T13:56:20.072 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.060+0000 7f2312f04700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f230c145bd0 con 0x7f230c0b81c0 2020-11-20T13:56:20.072 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.060+0000 7f2310ca0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f230c099f10 0x7f230c067640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:20.072 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.060+0000 7f2310ca0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f230c099f10 0x7f230c067640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54394/0 (socket says 172.21.15.40:54394) 2020-11-20T13:56:20.073 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.060+0000 7f2310ca0700 1 -- 172.21.15.40:0/424973140 learned_addr learned my addr 172.21.15.40:0/424973140 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:20.073 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.060+0000 7f23114a1700 1 --2- 172.21.15.40:0/424973140 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f230c0b81c0 0x7f230c1567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:20.073 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.060+0000 7f2303fff700 1 --2- 172.21.15.40:0/424973140 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f230c09bc10 0x7f230c1456e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:20.073 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.060+0000 7f2310ca0700 1 -- 172.21.15.40:0/424973140 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f230c0b81c0 msgr2=0x7f230c1567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:20.073 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.060+0000 7f2310ca0700 1 --2- 172.21.15.40:0/424973140 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f230c0b81c0 0x7f230c1567a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:20.074 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.060+0000 7f2310ca0700 1 -- 172.21.15.40:0/424973140 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f230c09bc10 msgr2=0x7f230c1456e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:20.074 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.060+0000 7f2310ca0700 1 --2- 172.21.15.40:0/424973140 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f230c09bc10 0x7f230c1456e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:20.074 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.060+0000 7f2310ca0700 1 -- 172.21.15.40:0/424973140 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f230c09f120 con 0x7f230c099f10 2020-11-20T13:56:20.074 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.060+0000 7f2310ca0700 1 --2- 172.21.15.40:0/424973140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f230c099f10 0x7f230c067640 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f230c0627d0 tx=0x7f2308016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:20.075 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.060+0000 7f2301ffb700 1 -- 172.21.15.40:0/424973140 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2308011dc0 con 0x7f230c099f10 2020-11-20T13:56:20.075 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.060+0000 7f2301ffb700 1 -- 172.21.15.40:0/424973140 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2308021440 con 0x7f230c099f10 2020-11-20T13:56:20.076 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.060+0000 7f2312f04700 1 -- 172.21.15.40:0/424973140 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f230c09f630 con 0x7f230c099f10 2020-11-20T13:56:20.077 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.060+0000 7f2301ffb700 1 -- 172.21.15.40:0/424973140 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23080217a0 con 0x7f230c099f10 2020-11-20T13:56:20.077 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.060+0000 7f2312f04700 1 -- 172.21.15.40:0/424973140 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f230c09eb60 con 0x7f230c099f10 2020-11-20T13:56:20.077 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.061+0000 7f2312f04700 1 -- 172.21.15.40:0/424973140 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f230c09f450 con 0x7f230c099f10 2020-11-20T13:56:20.078 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.062+0000 7f2301ffb700 1 -- 172.21.15.40:0/424973140 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f2308027020 con 0x7f230c099f10 2020-11-20T13:56:20.078 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.062+0000 7f2301ffb700 1 --2- 172.21.15.40:0/424973140 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f22ec051210 0x7f22ec0536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:20.078 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.062+0000 7f2301ffb700 1 -- 172.21.15.40:0/424973140 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f2308069cd0 con 0x7f230c099f10 2020-11-20T13:56:20.079 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.062+0000 7f2303fff700 1 --2- 172.21.15.40:0/424973140 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f22ec051210 0x7f22ec0536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:20.079 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.062+0000 7f2303fff700 1 --2- 172.21.15.40:0/424973140 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f22ec051210 0x7f22ec0536d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f22fc00d000 tx=0x7f22fc008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:20.079 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.065+0000 7f2301ffb700 1 -- 172.21.15.40:0/424973140 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f2308042430 con 0x7f230c099f10 2020-11-20T13:56:20.229 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.229+0000 7f2312f04700 1 -- 172.21.15.40:0/424973140 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f230c057050 con 0x7f230c099f10 2020-11-20T13:56:20.229 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.229+0000 7f2301ffb700 1 -- 172.21.15.40:0/424973140 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) v1 ==== 84+0+143 (secure 0 0 0) 0x7f2308015090 con 0x7f230c099f10 2020-11-20T13:56:20.230 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:56:20.230 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2020-11-20T13:56:20.230 INFO:teuthology.orchestra.run.smithi040.stdout: { 2020-11-20T13:56:20.230 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 1, 2020-11-20T13:56:20.231 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": "device_health_metrics" 2020-11-20T13:56:20.231 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2020-11-20T13:56:20.231 INFO:teuthology.orchestra.run.smithi040.stdout: { 2020-11-20T13:56:20.231 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 2, 2020-11-20T13:56:20.231 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": "rbd" 2020-11-20T13:56:20.232 INFO:teuthology.orchestra.run.smithi040.stdout: } 2020-11-20T13:56:20.232 INFO:teuthology.orchestra.run.smithi040.stdout:] 2020-11-20T13:56:20.233 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.231+0000 7f2312f04700 1 -- 172.21.15.40:0/424973140 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f22ec051210 msgr2=0x7f22ec0536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:20.233 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.231+0000 7f2312f04700 1 --2- 172.21.15.40:0/424973140 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f22ec051210 0x7f22ec0536d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f22fc00d000 tx=0x7f22fc008040).stop 2020-11-20T13:56:20.233 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.231+0000 7f2312f04700 1 -- 172.21.15.40:0/424973140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f230c099f10 msgr2=0x7f230c067640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:20.234 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.231+0000 7f2312f04700 1 --2- 172.21.15.40:0/424973140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f230c099f10 0x7f230c067640 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f230c0627d0 tx=0x7f2308016000).stop 2020-11-20T13:56:20.234 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.231+0000 7f2312f04700 1 -- 172.21.15.40:0/424973140 shutdown_connections 2020-11-20T13:56:20.234 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.231+0000 7f2312f04700 1 --2- 172.21.15.40:0/424973140 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f22ec051210 0x7f22ec0536d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:20.234 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.231+0000 7f2312f04700 1 --2- 172.21.15.40:0/424973140 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f230c0b81c0 0x7f230c1567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:20.234 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.231+0000 7f2312f04700 1 --2- 172.21.15.40:0/424973140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f230c099f10 0x7f230c067640 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:20.235 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.231+0000 7f2312f04700 1 --2- 172.21.15.40:0/424973140 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f230c09bc10 0x7f230c1456e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:20.238 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.231+0000 7f2312f04700 1 -- 172.21.15.40:0/424973140 >> 172.21.15.40:0/424973140 conn(0x7f230c0ba380 msgr2=0x7f230c13c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:20.238 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.231+0000 7f2312f04700 1 -- 172.21.15.40:0/424973140 shutdown_connections 2020-11-20T13:56:20.238 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.231+0000 7f2312f04700 1 -- 172.21.15.40:0/424973140 wait complete. 2020-11-20T13:56:20.241 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-11-20T13:56:20.241 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2020-11-20T13:56:20.359 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.359+0000 7fb750105700 1 Processor -- start 2020-11-20T13:56:20.360 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.359+0000 7fb750105700 1 -- start start 2020-11-20T13:56:20.366 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.360+0000 7fb750105700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb7480b5bf0 0x7fb74805f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:20.366 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.360+0000 7fb750105700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb7480b3960 0x7fb74814cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:20.367 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.360+0000 7fb750105700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7480b5fd0 con 0x7fb748144390 2020-11-20T13:56:20.367 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.360+0000 7fb750105700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7480b05f0 con 0x7fb7480b5bf0 2020-11-20T13:56:20.367 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.360+0000 7fb750105700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb748144870 con 0x7fb7480b3960 2020-11-20T13:56:20.367 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.360+0000 7fb74dea1700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb7480b5bf0 0x7fb74805f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:20.368 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.360+0000 7fb74d6a0700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb748144390 0x7fb74805f880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45000/0 (socket says 172.21.15.40:45000) 2020-11-20T13:56:20.368 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.360+0000 7fb74d6a0700 1 -- 172.21.15.40:0/1269978223 learned_addr learned my addr 172.21.15.40:0/1269978223 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:20.368 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.360+0000 7fb74dea1700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb7480b5bf0 0x7fb74805f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54402/0 (socket says 172.21.15.40:54402) 2020-11-20T13:56:20.369 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.361+0000 7fb74e6a2700 1 --2- 172.21.15.40:0/1269978223 >> v2:172.21.15.116:3300/0 conn(0x7fb7480b3960 0x7fb74814cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:20.369 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.361+0000 7fb74dea1700 1 -- 172.21.15.40:0/1269978223 >> v2:172.21.15.116:3300/0 conn(0x7fb7480b3960 msgr2=0x7fb74814cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:20.371 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.361+0000 7fb74dea1700 1 --2- 172.21.15.40:0/1269978223 >> v2:172.21.15.116:3300/0 conn(0x7fb7480b3960 0x7fb74814cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:20.371 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.361+0000 7fb74dea1700 1 -- 172.21.15.40:0/1269978223 >> v1:172.21.15.40:6789/0 conn(0x7fb748144390 legacy=0x7fb74805f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:20.371 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.361+0000 7fb74dea1700 1 -- 172.21.15.40:0/1269978223 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7480b0c30 con 0x7fb7480b5bf0 2020-11-20T13:56:20.372 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.361+0000 7fb74ce9f700 1 -- 172.21.15.40:0/1269978223 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2217633122 0 0) 0x7fb7480b5fd0 con 0x7fb748144390 2020-11-20T13:56:20.372 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.361+0000 7fb74dea1700 1 --2- 172.21.15.40:0/1269978223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7480b5bf0 0x7fb74805f390 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fb73c002f30 tx=0x7fb73c006d60).ready entity=mon.0 client_cookie=9bbf94f00e9a48ad server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:20.372 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.361+0000 7fb74ce9f700 1 -- 172.21.15.40:0/1269978223 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb73c0097f0 con 0x7fb7480b5bf0 2020-11-20T13:56:20.372 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.361+0000 7fb74ce9f700 1 -- 172.21.15.40:0/1269978223 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb73c009950 con 0x7fb7480b5bf0 2020-11-20T13:56:20.373 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.361+0000 7fb74ce9f700 1 -- 172.21.15.40:0/1269978223 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb73c009ac0 con 0x7fb7480b5bf0 2020-11-20T13:56:20.373 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.361+0000 7fb750105700 1 -- 172.21.15.40:0/1269978223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7480b5bf0 msgr2=0x7fb74805f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:20.373 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.361+0000 7fb750105700 1 --2- 172.21.15.40:0/1269978223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7480b5bf0 0x7fb74805f390 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fb73c002f30 tx=0x7fb73c006d60).stop 2020-11-20T13:56:20.373 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.361+0000 7fb750105700 1 -- 172.21.15.40:0/1269978223 shutdown_connections 2020-11-20T13:56:20.373 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.361+0000 7fb750105700 1 --2- 172.21.15.40:0/1269978223 >> v2:172.21.15.116:3300/0 conn(0x7fb7480b3960 0x7fb74814cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:20.374 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.361+0000 7fb750105700 1 --2- 172.21.15.40:0/1269978223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7480b5bf0 0x7fb74805f390 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:20.374 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.361+0000 7fb750105700 1 -- 172.21.15.40:0/1269978223 >> 172.21.15.40:0/1269978223 conn(0x7fb7480ba1d0 msgr2=0x7fb7480b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:20.374 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.362+0000 7fb750105700 1 -- 172.21.15.40:0/1269978223 shutdown_connections 2020-11-20T13:56:20.374 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.362+0000 7fb750105700 1 -- 172.21.15.40:0/1269978223 wait complete. 2020-11-20T13:56:20.374 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.362+0000 7fb750105700 1 Processor -- start 2020-11-20T13:56:20.375 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.362+0000 7fb750105700 1 -- start start 2020-11-20T13:56:20.375 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.363+0000 7fb750105700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb7480b3960 0x7fb748067290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:20.375 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.363+0000 7fb750105700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7480b5bf0 0x7fb748067780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:20.375 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.363+0000 7fb750105700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb748144390 0x7fb748137fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:20.375 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.363+0000 7fb750105700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7480afc20 con 0x7fb748144390 2020-11-20T13:56:20.376 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.363+0000 7fb750105700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb748063d60 con 0x7fb7480b5bf0 2020-11-20T13:56:20.376 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.363+0000 7fb750105700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb748064000 con 0x7fb7480b3960 2020-11-20T13:56:20.376 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.363+0000 7fb74e6a2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb748144390 0x7fb748137fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:20.376 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.363+0000 7fb74e6a2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb748144390 0x7fb748137fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54412/0 (socket says 172.21.15.40:54412) 2020-11-20T13:56:20.376 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.363+0000 7fb74e6a2700 1 -- 172.21.15.40:0/3848540345 learned_addr learned my addr 172.21.15.40:0/3848540345 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:20.377 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.363+0000 7fb74dea1700 1 --2- 172.21.15.40:0/3848540345 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb7480b3960 0x7fb748067290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:20.377 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.363+0000 7fb74d6a0700 1 --2- 172.21.15.40:0/3848540345 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7480b5bf0 0x7fb748067780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:20.377 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.364+0000 7fb74e6a2700 1 -- 172.21.15.40:0/3848540345 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb7480b3960 msgr2=0x7fb748067290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:20.377 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.364+0000 7fb74e6a2700 1 --2- 172.21.15.40:0/3848540345 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb7480b3960 0x7fb748067290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:20.377 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.364+0000 7fb74e6a2700 1 -- 172.21.15.40:0/3848540345 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7480b5bf0 msgr2=0x7fb748067780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:20.378 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.364+0000 7fb74e6a2700 1 --2- 172.21.15.40:0/3848540345 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7480b5bf0 0x7fb748067780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:20.378 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.364+0000 7fb74e6a2700 1 -- 172.21.15.40:0/3848540345 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7480642a0 con 0x7fb748144390 2020-11-20T13:56:20.378 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.364+0000 7fb74d6a0700 1 --2- 172.21.15.40:0/3848540345 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7480b5bf0 0x7fb748067780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:20.378 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.364+0000 7fb74e6a2700 1 --2- 172.21.15.40:0/3848540345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb748144390 0x7fb748137fa0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fb74000a000 tx=0x7fb740008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:20.378 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.364+0000 7fb73affd700 1 -- 172.21.15.40:0/3848540345 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb74000b9e0 con 0x7fb748144390 2020-11-20T13:56:20.379 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.364+0000 7fb73affd700 1 -- 172.21.15.40:0/3848540345 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb74000bb40 con 0x7fb748144390 2020-11-20T13:56:20.379 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.364+0000 7fb750105700 1 -- 172.21.15.40:0/3848540345 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb748064740 con 0x7fb748144390 2020-11-20T13:56:20.379 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.364+0000 7fb73affd700 1 -- 172.21.15.40:0/3848540345 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb74000bea0 con 0x7fb748144390 2020-11-20T13:56:20.379 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.364+0000 7fb750105700 1 -- 172.21.15.40:0/3848540345 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb748140d20 con 0x7fb748144390 2020-11-20T13:56:20.379 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.365+0000 7fb73affd700 1 -- 172.21.15.40:0/3848540345 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fb74001c460 con 0x7fb748144390 2020-11-20T13:56:20.380 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.365+0000 7fb750105700 1 -- 172.21.15.40:0/3848540345 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb748064430 con 0x7fb748144390 2020-11-20T13:56:20.380 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.365+0000 7fb73affd700 1 --2- 172.21.15.40:0/3848540345 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fb728051320 0x7fb7280537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:20.380 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.365+0000 7fb73affd700 1 -- 172.21.15.40:0/3848540345 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fb740069320 con 0x7fb748144390 2020-11-20T13:56:20.380 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.365+0000 7fb74dea1700 1 --2- 172.21.15.40:0/3848540345 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fb728051320 0x7fb7280537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:20.380 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.366+0000 7fb74dea1700 1 --2- 172.21.15.40:0/3848540345 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fb728051320 0x7fb7280537e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fb73c01b000 tx=0x7fb73c016000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:20.381 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.369+0000 7fb73affd700 1 -- 172.21.15.40:0/3848540345 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fb740041970 con 0x7fb748144390 2020-11-20T13:56:20.530 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:20.528+0000 7fb750105700 1 -- 172.21.15.40:0/3848540345 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fb748057050 con 0x7fb748144390 2020-11-20T13:56:21.019 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.019+0000 7fb73affd700 1 -- 172.21.15.40:0/3848540345 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v19) v1 ==== 133+0+0 (secure 0 0 0) 0x7fb740044f90 con 0x7fb748144390 2020-11-20T13:56:21.022 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.022+0000 7fb750105700 1 -- 172.21.15.40:0/3848540345 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fb728051320 msgr2=0x7fb7280537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:21.022 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.022+0000 7fb750105700 1 --2- 172.21.15.40:0/3848540345 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fb728051320 0x7fb7280537e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fb73c01b000 tx=0x7fb73c016000).stop 2020-11-20T13:56:21.023 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.022+0000 7fb750105700 1 -- 172.21.15.40:0/3848540345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb748144390 msgr2=0x7fb748137fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:21.023 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.022+0000 7fb750105700 1 --2- 172.21.15.40:0/3848540345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb748144390 0x7fb748137fa0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fb74000a000 tx=0x7fb740008040).stop 2020-11-20T13:56:21.023 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.022+0000 7fb750105700 1 -- 172.21.15.40:0/3848540345 shutdown_connections 2020-11-20T13:56:21.023 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.022+0000 7fb750105700 1 --2- 172.21.15.40:0/3848540345 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fb728051320 0x7fb7280537e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:21.023 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.022+0000 7fb750105700 1 --2- 172.21.15.40:0/3848540345 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb7480b3960 0x7fb748067290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:21.024 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.022+0000 7fb750105700 1 --2- 172.21.15.40:0/3848540345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb748144390 0x7fb748137fa0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:21.024 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.022+0000 7fb750105700 1 --2- 172.21.15.40:0/3848540345 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7480b5bf0 0x7fb748067780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:21.024 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.022+0000 7fb750105700 1 -- 172.21.15.40:0/3848540345 >> 172.21.15.40:0/3848540345 conn(0x7fb7480ba1d0 msgr2=0x7fb7480b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:21.024 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.022+0000 7fb750105700 1 -- 172.21.15.40:0/3848540345 shutdown_connections 2020-11-20T13:56:21.024 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.022+0000 7fb750105700 1 -- 172.21.15.40:0/3848540345 wait complete. 2020-11-20T13:56:21.025 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' created 2020-11-20T13:56:21.036 INFO:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2020-11-20T13:56:21.036 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd erasure-code-profile set cephfs_data m=2 k=2 crush-failure-domain=osd 2020-11-20T13:56:21.160 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.160+0000 7fb3b1309700 1 Processor -- start 2020-11-20T13:56:21.161 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.160+0000 7fb3b1309700 1 -- start start 2020-11-20T13:56:21.163 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.161+0000 7fb3b1309700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb3ac0b1b60 0x7fb3ac144bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:21.163 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.161+0000 7fb3b1309700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3ac0b0d10 con 0x7fb3ac0b31f0 2020-11-20T13:56:21.163 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.161+0000 7fb3b1309700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3ac0b0fb0 con 0x7fb3ac0b6c80 2020-11-20T13:56:21.164 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.161+0000 7fb3b1309700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3ac0b7060 con 0x7fb3ac0b1b60 2020-11-20T13:56:21.164 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.161+0000 7fb3ab7fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb3ac0b1b60 0x7fb3ac144bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:21.164 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.161+0000 7fb3aa7fc700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb3ac0b31f0 0x7fb3ac0b1450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45012/0 (socket says 172.21.15.40:45012) 2020-11-20T13:56:21.164 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.162+0000 7fb3aa7fc700 1 -- 172.21.15.40:0/4155335637 learned_addr learned my addr 172.21.15.40:0/4155335637 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:21.164 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.162+0000 7fb3ab7fe700 1 -- 172.21.15.40:0/4155335637 >> v1:172.21.15.116:6789/0 conn(0x7fb3ac0b6c80 legacy=0x7fb3ac0b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:21.165 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.162+0000 7fb3a9ffb700 1 -- 172.21.15.40:0/4155335637 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2303752741 0 0) 0x7fb3ac0b0d10 con 0x7fb3ac0b31f0 2020-11-20T13:56:21.165 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.162+0000 7fb3ab7fe700 1 -- 172.21.15.40:0/4155335637 >> v1:172.21.15.40:6789/0 conn(0x7fb3ac0b31f0 legacy=0x7fb3ac0b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:21.165 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.162+0000 7fb3ab7fe700 1 -- 172.21.15.40:0/4155335637 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3ac0b4800 con 0x7fb3ac0b1b60 2020-11-20T13:56:21.165 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.162+0000 7fb3ab7fe700 1 --2- 172.21.15.40:0/4155335637 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3ac0b1b60 0x7fb3ac144bd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fb3a4002f30 tx=0x7fb3a4006dc0).ready entity=mon.0 client_cookie=722e0e7192b5feac server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:21.165 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.162+0000 7fb3a9ffb700 1 -- 172.21.15.40:0/4155335637 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3a40097f0 con 0x7fb3ac0b1b60 2020-11-20T13:56:21.166 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.162+0000 7fb3a9ffb700 1 -- 172.21.15.40:0/4155335637 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3a4009950 con 0x7fb3ac0b1b60 2020-11-20T13:56:21.166 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.162+0000 7fb3a9ffb700 1 -- 172.21.15.40:0/4155335637 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3a4009cb0 con 0x7fb3ac0b1b60 2020-11-20T13:56:21.166 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.162+0000 7fb3b1309700 1 -- 172.21.15.40:0/4155335637 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3ac0b1b60 msgr2=0x7fb3ac144bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:21.166 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.162+0000 7fb3b1309700 1 --2- 172.21.15.40:0/4155335637 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3ac0b1b60 0x7fb3ac144bd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fb3a4002f30 tx=0x7fb3a4006dc0).stop 2020-11-20T13:56:21.167 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.163+0000 7fb3b1309700 1 -- 172.21.15.40:0/4155335637 shutdown_connections 2020-11-20T13:56:21.167 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.163+0000 7fb3b1309700 1 --2- 172.21.15.40:0/4155335637 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3ac0b1b60 0x7fb3ac144bd0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:21.167 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.163+0000 7fb3b1309700 1 -- 172.21.15.40:0/4155335637 >> 172.21.15.40:0/4155335637 conn(0x7fb3ac0ba1d0 msgr2=0x7fb3ac0b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:21.167 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.163+0000 7fb3b1309700 1 -- 172.21.15.40:0/4155335637 shutdown_connections 2020-11-20T13:56:21.167 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.163+0000 7fb3b1309700 1 -- 172.21.15.40:0/4155335637 wait complete. 2020-11-20T13:56:21.170 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.164+0000 7fb3b1309700 1 Processor -- start 2020-11-20T13:56:21.170 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.164+0000 7fb3b1309700 1 -- start start 2020-11-20T13:56:21.170 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.164+0000 7fb3b1309700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb3ac0b1b60 0x7fb3ac12b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:21.171 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.164+0000 7fb3b1309700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3ac0b31f0 0x7fb3ac156730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:21.171 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.164+0000 7fb3b1309700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb3ac0b6c80 0x7fb3ac156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:21.171 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.164+0000 7fb3b1309700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac066f40 con 0x7fb3ac0b31f0 2020-11-20T13:56:21.171 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.164+0000 7fb3b1309700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac0671e0 con 0x7fb3ac0b1b60 2020-11-20T13:56:21.171 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.164+0000 7fb3b1309700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac067480 con 0x7fb3ac0b6c80 2020-11-20T13:56:21.174 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.164+0000 7fb3aa7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3ac0b31f0 0x7fb3ac156730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:21.174 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.165+0000 7fb3aa7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3ac0b31f0 0x7fb3ac156730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54424/0 (socket says 172.21.15.40:54424) 2020-11-20T13:56:21.175 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.165+0000 7fb3aa7fc700 1 -- 172.21.15.40:0/1148155104 learned_addr learned my addr 172.21.15.40:0/1148155104 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:21.175 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.165+0000 7fb3ab7fe700 1 --2- 172.21.15.40:0/1148155104 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb3ac0b6c80 0x7fb3ac156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:21.176 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.165+0000 7fb3aaffd700 1 --2- 172.21.15.40:0/1148155104 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb3ac0b1b60 0x7fb3ac12b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:21.176 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.165+0000 7fb3aa7fc700 1 -- 172.21.15.40:0/1148155104 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb3ac0b6c80 msgr2=0x7fb3ac156c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:21.177 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.165+0000 7fb3aa7fc700 1 --2- 172.21.15.40:0/1148155104 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb3ac0b6c80 0x7fb3ac156c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:21.177 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.165+0000 7fb3aa7fc700 1 -- 172.21.15.40:0/1148155104 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb3ac0b1b60 msgr2=0x7fb3ac12b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:21.180 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.165+0000 7fb3aa7fc700 1 --2- 172.21.15.40:0/1148155104 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb3ac0b1b60 0x7fb3ac12b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:21.180 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.165+0000 7fb3aa7fc700 1 -- 172.21.15.40:0/1148155104 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3ac062920 con 0x7fb3ac0b31f0 2020-11-20T13:56:21.180 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.165+0000 7fb3ab7fe700 1 --2- 172.21.15.40:0/1148155104 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb3ac0b6c80 0x7fb3ac156c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:21.180 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.165+0000 7fb3aaffd700 1 --2- 172.21.15.40:0/1148155104 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb3ac0b1b60 0x7fb3ac12b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:21.181 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.165+0000 7fb3aa7fc700 1 --2- 172.21.15.40:0/1148155104 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3ac0b31f0 0x7fb3ac156730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fb39c000f00 tx=0x7fb39c007d30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:21.181 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.165+0000 7fb39bfff700 1 -- 172.21.15.40:0/1148155104 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb39c005b70 con 0x7fb3ac0b31f0 2020-11-20T13:56:21.181 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.165+0000 7fb39bfff700 1 -- 172.21.15.40:0/1148155104 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb39c005cd0 con 0x7fb3ac0b31f0 2020-11-20T13:56:21.181 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.165+0000 7fb3b1309700 1 -- 172.21.15.40:0/1148155104 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3ac157110 con 0x7fb3ac0b31f0 2020-11-20T13:56:21.181 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.165+0000 7fb39bfff700 1 -- 172.21.15.40:0/1148155104 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb39c007300 con 0x7fb3ac0b31f0 2020-11-20T13:56:21.182 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.165+0000 7fb3b1309700 1 -- 172.21.15.40:0/1148155104 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3ac1532d0 con 0x7fb3ac0b31f0 2020-11-20T13:56:21.182 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.166+0000 7fb3b1309700 1 -- 172.21.15.40:0/1148155104 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3ac067640 con 0x7fb3ac0b31f0 2020-11-20T13:56:21.182 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.166+0000 7fb39bfff700 1 -- 172.21.15.40:0/1148155104 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fb39c010030 con 0x7fb3ac0b31f0 2020-11-20T13:56:21.182 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.166+0000 7fb39bfff700 1 --2- 172.21.15.40:0/1148155104 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fb388051290 0x7fb388053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:21.182 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.167+0000 7fb39bfff700 1 -- 172.21.15.40:0/1148155104 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5607+0+0 (secure 0 0 0) 0x7fb39c068e10 con 0x7fb3ac0b31f0 2020-11-20T13:56:21.183 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.167+0000 7fb3aaffd700 1 --2- 172.21.15.40:0/1148155104 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fb388051290 0x7fb388053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:21.183 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.167+0000 7fb3aaffd700 1 --2- 172.21.15.40:0/1148155104 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fb388051290 0x7fb388053750 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fb3a000b000 tx=0x7fb3a0009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:21.183 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.170+0000 7fb39bfff700 1 -- 172.21.15.40:0/1148155104 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fb39c041300 con 0x7fb3ac0b31f0 2020-11-20T13:56:21.348 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:21.347+0000 7fb3b1309700 1 -- 172.21.15.40:0/1148155104 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7fb3ac057050 con 0x7fb3ac0b31f0 2020-11-20T13:56:22.025 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.025+0000 7fb39bfff700 1 -- 172.21.15.40:0/1148155104 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) v1 ==== 154+0+0 (secure 0 0 0) 0x7fb39c044920 con 0x7fb3ac0b31f0 2020-11-20T13:56:22.029 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.029+0000 7fb3b1309700 1 -- 172.21.15.40:0/1148155104 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fb388051290 msgr2=0x7fb388053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:22.030 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.029+0000 7fb3b1309700 1 --2- 172.21.15.40:0/1148155104 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fb388051290 0x7fb388053750 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fb3a000b000 tx=0x7fb3a0009040).stop 2020-11-20T13:56:22.030 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.029+0000 7fb3b1309700 1 -- 172.21.15.40:0/1148155104 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3ac0b31f0 msgr2=0x7fb3ac156730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:22.030 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.029+0000 7fb3b1309700 1 --2- 172.21.15.40:0/1148155104 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3ac0b31f0 0x7fb3ac156730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fb39c000f00 tx=0x7fb39c007d30).stop 2020-11-20T13:56:22.030 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.029+0000 7fb3b1309700 1 -- 172.21.15.40:0/1148155104 shutdown_connections 2020-11-20T13:56:22.031 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.029+0000 7fb3b1309700 1 --2- 172.21.15.40:0/1148155104 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fb388051290 0x7fb388053750 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:22.031 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.029+0000 7fb3b1309700 1 --2- 172.21.15.40:0/1148155104 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb3ac0b6c80 0x7fb3ac156c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:22.031 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.029+0000 7fb3b1309700 1 --2- 172.21.15.40:0/1148155104 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb3ac0b1b60 0x7fb3ac12b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:22.031 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.029+0000 7fb3b1309700 1 --2- 172.21.15.40:0/1148155104 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3ac0b31f0 0x7fb3ac156730 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:22.032 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.029+0000 7fb3b1309700 1 -- 172.21.15.40:0/1148155104 >> 172.21.15.40:0/1148155104 conn(0x7fb3ac0ba1d0 msgr2=0x7fb3ac146fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:22.032 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.029+0000 7fb3b1309700 1 -- 172.21.15.40:0/1148155104 shutdown_connections 2020-11-20T13:56:22.032 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.029+0000 7fb3b1309700 1 -- 172.21.15.40:0/1148155104 wait complete. 2020-11-20T13:56:22.047 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 erasure cephfs_data 2020-11-20T13:56:22.174 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.174+0000 7f3209800700 1 Processor -- start 2020-11-20T13:56:22.175 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.174+0000 7f3209800700 1 -- start start 2020-11-20T13:56:22.184 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.175+0000 7f3209800700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f32041480a0 0x7f32040b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:22.184 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.175+0000 7f3209800700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f32041444a0 0x7f32040ad430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:22.184 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.175+0000 7f3209800700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32041449a0 con 0x7f3204144020 2020-11-20T13:56:22.184 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.175+0000 7f3209800700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32040ad040 con 0x7f32041444a0 2020-11-20T13:56:22.185 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.175+0000 7f3209800700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3204144c40 con 0x7f32041480a0 2020-11-20T13:56:22.185 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.175+0000 7f3208fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f32041444a0 0x7f32040ad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:22.185 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.175+0000 7f3208fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f32041444a0 0x7f32040ad430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54430/0 (socket says 172.21.15.40:54430) 2020-11-20T13:56:22.185 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.175+0000 7f3208fff700 1 -- 172.21.15.40:0/3197764600 learned_addr learned my addr 172.21.15.40:0/3197764600 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:22.186 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.175+0000 7f3203fff700 1 --2- 172.21.15.40:0/3197764600 >> v2:172.21.15.116:3300/0 conn(0x7f32041480a0 0x7f32040b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:22.186 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.175+0000 7f3208fff700 1 -- 172.21.15.40:0/3197764600 >> v2:172.21.15.116:3300/0 conn(0x7f32041480a0 msgr2=0x7f32040b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:22.186 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.175+0000 7f3208fff700 1 --2- 172.21.15.40:0/3197764600 >> v2:172.21.15.116:3300/0 conn(0x7f32041480a0 0x7f32040b05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:22.186 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.175+0000 7f3208fff700 1 -- 172.21.15.40:0/3197764600 >> v1:172.21.15.116:6789/0 conn(0x7f3204144020 legacy=0x7f32040b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:22.186 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.175+0000 7f3208fff700 1 -- 172.21.15.40:0/3197764600 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32040b56e0 con 0x7f32041444a0 2020-11-20T13:56:22.187 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.175+0000 7f3208fff700 1 --2- 172.21.15.40:0/3197764600 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32041444a0 0x7f32040ad430 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f31fc0032b0 tx=0x7f31fc009270).ready entity=mon.0 client_cookie=e1c0c4d0e6734f78 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:22.187 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.176+0000 7f3202ffd700 1 -- 172.21.15.40:0/3197764600 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31fc00bc00 con 0x7f32041444a0 2020-11-20T13:56:22.187 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.176+0000 7f3202ffd700 1 -- 172.21.15.40:0/3197764600 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31fc00bd60 con 0x7f32041444a0 2020-11-20T13:56:22.187 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.176+0000 7f3202ffd700 1 -- 172.21.15.40:0/3197764600 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31fc011400 con 0x7f32041444a0 2020-11-20T13:56:22.188 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.176+0000 7f3209800700 1 -- 172.21.15.40:0/3197764600 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32041444a0 msgr2=0x7f32040ad430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:22.188 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.176+0000 7f3209800700 1 --2- 172.21.15.40:0/3197764600 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32041444a0 0x7f32040ad430 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f31fc0032b0 tx=0x7f31fc009270).stop 2020-11-20T13:56:22.188 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.176+0000 7f3209800700 1 -- 172.21.15.40:0/3197764600 shutdown_connections 2020-11-20T13:56:22.188 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.176+0000 7f3209800700 1 --2- 172.21.15.40:0/3197764600 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32041444a0 0x7f32040ad430 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:22.189 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.176+0000 7f3209800700 1 --2- 172.21.15.40:0/3197764600 >> v2:172.21.15.116:3300/0 conn(0x7f32041480a0 0x7f32040b05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:22.189 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.176+0000 7f3209800700 1 -- 172.21.15.40:0/3197764600 >> 172.21.15.40:0/3197764600 conn(0x7f32040b9520 msgr2=0x7f32040b9930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:22.190 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.176+0000 7f3209800700 1 -- 172.21.15.40:0/3197764600 shutdown_connections 2020-11-20T13:56:22.190 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.176+0000 7f3209800700 1 -- 172.21.15.40:0/3197764600 wait complete. 2020-11-20T13:56:22.190 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.177+0000 7f3209800700 1 Processor -- start 2020-11-20T13:56:22.190 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.177+0000 7f3209800700 1 -- start start 2020-11-20T13:56:22.191 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.178+0000 7f3209800700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3204144020 0x7f320413fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:22.192 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.178+0000 7f3209800700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32041444a0 0x7f3204129b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:22.192 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.178+0000 7f3209800700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32041480a0 0x7f320412a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:22.192 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.178+0000 7f3209800700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320413a630 con 0x7f32041444a0 2020-11-20T13:56:22.192 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.178+0000 7f3209800700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320413ff00 con 0x7f32041480a0 2020-11-20T13:56:22.193 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.178+0000 7f3209800700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3204143a60 con 0x7f3204144020 2020-11-20T13:56:22.193 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.178+0000 7f32037fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32041444a0 0x7f3204129b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:22.193 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.178+0000 7f32037fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32041444a0 0x7f3204129b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54436/0 (socket says 172.21.15.40:54436) 2020-11-20T13:56:22.193 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.178+0000 7f32037fe700 1 -- 172.21.15.40:0/806725660 learned_addr learned my addr 172.21.15.40:0/806725660 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:22.194 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.178+0000 7f3208fff700 1 --2- 172.21.15.40:0/806725660 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32041480a0 0x7f320412a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:22.194 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.178+0000 7f3203fff700 1 --2- 172.21.15.40:0/806725660 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3204144020 0x7f320413fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:22.194 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.179+0000 7f32037fe700 1 -- 172.21.15.40:0/806725660 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3204144020 msgr2=0x7f320413fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:22.194 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.179+0000 7f32037fe700 1 --2- 172.21.15.40:0/806725660 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3204144020 0x7f320413fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:22.194 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.179+0000 7f32037fe700 1 -- 172.21.15.40:0/806725660 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32041480a0 msgr2=0x7f320412a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:22.195 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.179+0000 7f32037fe700 1 --2- 172.21.15.40:0/806725660 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32041480a0 0x7f320412a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:22.195 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.179+0000 7f32037fe700 1 -- 172.21.15.40:0/806725660 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3204143320 con 0x7f32041444a0 2020-11-20T13:56:22.195 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.179+0000 7f32037fe700 1 --2- 172.21.15.40:0/806725660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32041444a0 0x7f3204129b10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f32040687b0 tx=0x7f31f8007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:22.195 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.179+0000 7f32017fa700 1 -- 172.21.15.40:0/806725660 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31f8007240 con 0x7f32041444a0 2020-11-20T13:56:22.195 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.179+0000 7f32017fa700 1 -- 172.21.15.40:0/806725660 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f80073a0 con 0x7f32041444a0 2020-11-20T13:56:22.196 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.179+0000 7f3209800700 1 -- 172.21.15.40:0/806725660 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32041435c0 con 0x7f32041444a0 2020-11-20T13:56:22.196 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.179+0000 7f32017fa700 1 -- 172.21.15.40:0/806725660 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31f80064e0 con 0x7f32041444a0 2020-11-20T13:56:22.196 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.179+0000 7f3209800700 1 -- 172.21.15.40:0/806725660 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32040af8e0 con 0x7f32041444a0 2020-11-20T13:56:22.196 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.180+0000 7f3209800700 1 -- 172.21.15.40:0/806725660 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3204143750 con 0x7f32041444a0 2020-11-20T13:56:22.196 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.180+0000 7f32017fa700 1 -- 172.21.15.40:0/806725660 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f31f8010070 con 0x7f32041444a0 2020-11-20T13:56:22.197 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.180+0000 7f32017fa700 1 --2- 172.21.15.40:0/806725660 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f31e4051320 0x7f31e40537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:22.197 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.180+0000 7f32017fa700 1 -- 172.21.15.40:0/806725660 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5863+0+0 (secure 0 0 0) 0x7f31f8068490 con 0x7f32041444a0 2020-11-20T13:56:22.197 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.180+0000 7f3203fff700 1 --2- 172.21.15.40:0/806725660 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f31e4051320 0x7f31e40537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:22.197 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.181+0000 7f3203fff700 1 --2- 172.21.15.40:0/806725660 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f31e4051320 0x7f31e40537e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f31f400c000 tx=0x7f31f4008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:22.198 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.184+0000 7f32017fa700 1 -- 172.21.15.40:0/806725660 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f31f8040810 con 0x7f32041444a0 2020-11-20T13:56:22.350 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:22.350+0000 7f3209800700 1 -- 172.21.15.40:0/806725660 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8, "pool_type": "erasure", "erasure_code_profile": "cephfs_data"} v 0) v1 -- 0x7f3204057050 con 0x7f32041444a0 2020-11-20T13:56:24.035 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.035+0000 7f32017fa700 1 -- 172.21.15.40:0/806725660 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8, "pool_type": "erasure", "erasure_code_profile": "cephfs_data"}]=0 pool 'cephfs_data' created v22) v1 ==== 188+0+0 (secure 0 0 0) 0x7f31f8043e30 con 0x7f32041444a0 2020-11-20T13:56:24.038 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.037+0000 7f3209800700 1 -- 172.21.15.40:0/806725660 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f31e4051320 msgr2=0x7f31e40537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:24.039 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.037+0000 7f3209800700 1 --2- 172.21.15.40:0/806725660 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f31e4051320 0x7f31e40537e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f31f400c000 tx=0x7f31f4008040).stop 2020-11-20T13:56:24.039 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.037+0000 7f3209800700 1 -- 172.21.15.40:0/806725660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32041444a0 msgr2=0x7f3204129b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:24.040 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.037+0000 7f3209800700 1 --2- 172.21.15.40:0/806725660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32041444a0 0x7f3204129b10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f32040687b0 tx=0x7f31f8007be0).stop 2020-11-20T13:56:24.040 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.037+0000 7f3209800700 1 -- 172.21.15.40:0/806725660 shutdown_connections 2020-11-20T13:56:24.040 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.037+0000 7f3209800700 1 --2- 172.21.15.40:0/806725660 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f31e4051320 0x7f31e40537e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:24.041 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.037+0000 7f3209800700 1 --2- 172.21.15.40:0/806725660 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3204144020 0x7f320413fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:24.041 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.037+0000 7f3209800700 1 --2- 172.21.15.40:0/806725660 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32041480a0 0x7f320412a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:24.042 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.037+0000 7f3209800700 1 --2- 172.21.15.40:0/806725660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32041444a0 0x7f3204129b10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:24.042 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.037+0000 7f3209800700 1 -- 172.21.15.40:0/806725660 >> 172.21.15.40:0/806725660 conn(0x7f32040b9520 msgr2=0x7f32040b3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:24.042 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.038+0000 7f3209800700 1 -- 172.21.15.40:0/806725660 shutdown_connections 2020-11-20T13:56:24.043 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.038+0000 7f3209800700 1 -- 172.21.15.40:0/806725660 wait complete. 2020-11-20T13:56:24.043 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' created 2020-11-20T13:56:24.053 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool set cephfs_data allow_ec_overwrites true 2020-11-20T13:56:24.179 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.179+0000 7fb9cae72700 1 Processor -- start 2020-11-20T13:56:24.179 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.179+0000 7fb9cae72700 1 -- start start 2020-11-20T13:56:24.181 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.180+0000 7fb9cae72700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb9c40b31f0 0x7fb9c4144bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:24.182 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.180+0000 7fb9cae72700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb9c40b1450 0x7fb9c40b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:24.182 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.180+0000 7fb9cae72700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9c40b1dd0 con 0x7fb9c40b6c80 2020-11-20T13:56:24.182 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.180+0000 7fb9cae72700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9c40b0d10 con 0x7fb9c40b31f0 2020-11-20T13:56:24.182 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.180+0000 7fb9cae72700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9c40b0fb0 con 0x7fb9c40b1450 2020-11-20T13:56:24.183 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.180+0000 7fb9c3fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb9c40b31f0 0x7fb9c4144bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:24.183 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.180+0000 7fb9c3fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb9c40b31f0 0x7fb9c4144bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54450/0 (socket says 172.21.15.40:54450) 2020-11-20T13:56:24.183 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.180+0000 7fb9c3fff700 1 -- 172.21.15.40:0/26466825 learned_addr learned my addr 172.21.15.40:0/26466825 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:24.183 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.180+0000 7fb9c940f700 1 --2- 172.21.15.40:0/26466825 >> v2:172.21.15.116:3300/0 conn(0x7fb9c40b1450 0x7fb9c40b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:24.183 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.180+0000 7fb9c3fff700 1 -- 172.21.15.40:0/26466825 >> v2:172.21.15.116:3300/0 conn(0x7fb9c40b1450 msgr2=0x7fb9c40b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:24.184 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.180+0000 7fb9c3fff700 1 --2- 172.21.15.40:0/26466825 >> v2:172.21.15.116:3300/0 conn(0x7fb9c40b1450 0x7fb9c40b18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:24.184 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.180+0000 7fb9c3fff700 1 -- 172.21.15.40:0/26466825 >> v1:172.21.15.116:6789/0 conn(0x7fb9c40b6c80 legacy=0x7fb9c40b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:24.184 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.180+0000 7fb9c3fff700 1 -- 172.21.15.40:0/26466825 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c40b7060 con 0x7fb9c40b31f0 2020-11-20T13:56:24.184 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.180+0000 7fb9c3fff700 1 --2- 172.21.15.40:0/26466825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9c40b31f0 0x7fb9c4144bd0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fb9b8002f30 tx=0x7fb9b8006e80).ready entity=mon.0 client_cookie=868822588cef8c9f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:24.184 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.180+0000 7fb9c37fe700 1 -- 172.21.15.40:0/26466825 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9b80097f0 con 0x7fb9c40b31f0 2020-11-20T13:56:24.185 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.180+0000 7fb9c37fe700 1 -- 172.21.15.40:0/26466825 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b8009950 con 0x7fb9c40b31f0 2020-11-20T13:56:24.185 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.181+0000 7fb9c37fe700 1 -- 172.21.15.40:0/26466825 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9b8009cb0 con 0x7fb9c40b31f0 2020-11-20T13:56:24.185 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.181+0000 7fb9cae72700 1 -- 172.21.15.40:0/26466825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9c40b31f0 msgr2=0x7fb9c4144bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:24.185 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.181+0000 7fb9cae72700 1 --2- 172.21.15.40:0/26466825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9c40b31f0 0x7fb9c4144bd0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fb9b8002f30 tx=0x7fb9b8006e80).stop 2020-11-20T13:56:24.186 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.181+0000 7fb9cae72700 1 -- 172.21.15.40:0/26466825 shutdown_connections 2020-11-20T13:56:24.186 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.181+0000 7fb9cae72700 1 --2- 172.21.15.40:0/26466825 >> v2:172.21.15.116:3300/0 conn(0x7fb9c40b1450 0x7fb9c40b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:24.186 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.181+0000 7fb9cae72700 1 --2- 172.21.15.40:0/26466825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9c40b31f0 0x7fb9c4144bd0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:24.186 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.181+0000 7fb9cae72700 1 -- 172.21.15.40:0/26466825 >> 172.21.15.40:0/26466825 conn(0x7fb9c40ba1d0 msgr2=0x7fb9c40b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:24.186 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.181+0000 7fb9cae72700 1 -- 172.21.15.40:0/26466825 shutdown_connections 2020-11-20T13:56:24.187 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.181+0000 7fb9cae72700 1 -- 172.21.15.40:0/26466825 wait complete. 2020-11-20T13:56:24.190 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.182+0000 7fb9cae72700 1 Processor -- start 2020-11-20T13:56:24.190 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.182+0000 7fb9cae72700 1 -- start start 2020-11-20T13:56:24.190 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.183+0000 7fb9cae72700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb9c40b1450 0x7fb9c412bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:24.190 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.183+0000 7fb9cae72700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9c40b31f0 0x7fb9c41298a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:24.191 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.183+0000 7fb9cae72700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9c40b6c80 0x7fb9c4129d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:24.191 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.183+0000 7fb9cae72700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c4156380 con 0x7fb9c40b31f0 2020-11-20T13:56:24.191 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.183+0000 7fb9cae72700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c412a280 con 0x7fb9c40b6c80 2020-11-20T13:56:24.191 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.183+0000 7fb9cae72700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c4143a80 con 0x7fb9c40b1450 2020-11-20T13:56:24.191 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.183+0000 7fb9c3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9c40b31f0 0x7fb9c41298a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:24.192 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.183+0000 7fb9c3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9c40b31f0 0x7fb9c41298a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54454/0 (socket says 172.21.15.40:54454) 2020-11-20T13:56:24.192 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.183+0000 7fb9c3fff700 1 -- 172.21.15.40:0/639891357 learned_addr learned my addr 172.21.15.40:0/639891357 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:24.192 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.183+0000 7fb9c940f700 1 --2- 172.21.15.40:0/639891357 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9c40b6c80 0x7fb9c4129d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:24.192 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.183+0000 7fb9c8c0e700 1 --2- 172.21.15.40:0/639891357 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb9c40b1450 0x7fb9c412bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:24.192 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.183+0000 7fb9c3fff700 1 -- 172.21.15.40:0/639891357 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb9c40b1450 msgr2=0x7fb9c412bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:24.193 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.183+0000 7fb9c3fff700 1 --2- 172.21.15.40:0/639891357 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb9c40b1450 0x7fb9c412bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:24.193 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.183+0000 7fb9c3fff700 1 -- 172.21.15.40:0/639891357 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9c40b6c80 msgr2=0x7fb9c4129d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:24.193 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.183+0000 7fb9c3fff700 1 --2- 172.21.15.40:0/639891357 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9c40b6c80 0x7fb9c4129d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:24.193 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.183+0000 7fb9c3fff700 1 -- 172.21.15.40:0/639891357 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c415a540 con 0x7fb9c40b31f0 2020-11-20T13:56:24.194 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.183+0000 7fb9c3fff700 1 --2- 172.21.15.40:0/639891357 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9c40b31f0 0x7fb9c41298a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fb9b8002f30 tx=0x7fb9b8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:24.194 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.184+0000 7fb9c1ffb700 1 -- 172.21.15.40:0/639891357 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9b8011bf0 con 0x7fb9c40b31f0 2020-11-20T13:56:24.194 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.184+0000 7fb9c1ffb700 1 -- 172.21.15.40:0/639891357 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b8011d50 con 0x7fb9c40b31f0 2020-11-20T13:56:24.194 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.184+0000 7fb9cae72700 1 -- 172.21.15.40:0/639891357 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9c415a7e0 con 0x7fb9c40b31f0 2020-11-20T13:56:24.194 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.184+0000 7fb9c1ffb700 1 -- 172.21.15.40:0/639891357 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9b801e650 con 0x7fb9c40b31f0 2020-11-20T13:56:24.195 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.184+0000 7fb9cae72700 1 -- 172.21.15.40:0/639891357 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9c40630b0 con 0x7fb9c40b31f0 2020-11-20T13:56:24.195 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.185+0000 7fb9cae72700 1 -- 172.21.15.40:0/639891357 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9c4062cc0 con 0x7fb9c40b31f0 2020-11-20T13:56:24.195 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.185+0000 7fb9c1ffb700 1 -- 172.21.15.40:0/639891357 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fb9b8025030 con 0x7fb9c40b31f0 2020-11-20T13:56:24.195 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.185+0000 7fb9c1ffb700 1 --2- 172.21.15.40:0/639891357 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fb9a4051320 0x7fb9a40537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:24.195 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.185+0000 7fb9c1ffb700 1 -- 172.21.15.40:0/639891357 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6315+0+0 (secure 0 0 0) 0x7fb9b806a340 con 0x7fb9c40b31f0 2020-11-20T13:56:24.196 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.185+0000 7fb9c8c0e700 1 --2- 172.21.15.40:0/639891357 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fb9a4051320 0x7fb9a40537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:24.196 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.186+0000 7fb9c8c0e700 1 --2- 172.21.15.40:0/639891357 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fb9a4051320 0x7fb9a40537e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fb9bc00d000 tx=0x7fb9bc007e40).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:24.196 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.189+0000 7fb9c1ffb700 1 -- 172.21.15.40:0/639891357 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fb9b80425d0 con 0x7fb9c40b31f0 2020-11-20T13:56:24.369 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:24.369+0000 7fb9cae72700 1 -- 172.21.15.40:0/639891357 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fb9c415a970 con 0x7fb9c40b31f0 2020-11-20T13:56:25.039 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.039+0000 7fb9c1ffb700 1 -- 172.21.15.40:0/639891357 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) v1 ==== 166+0+0 (secure 0 0 0) 0x7fb9b80230c0 con 0x7fb9c40b31f0 2020-11-20T13:56:25.046 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.045+0000 7fb9cae72700 1 -- 172.21.15.40:0/639891357 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fb9a4051320 msgr2=0x7fb9a40537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:25.046 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.045+0000 7fb9cae72700 1 --2- 172.21.15.40:0/639891357 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fb9a4051320 0x7fb9a40537e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fb9bc00d000 tx=0x7fb9bc007e40).stop 2020-11-20T13:56:25.046 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.046+0000 7fb9cae72700 1 -- 172.21.15.40:0/639891357 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9c40b31f0 msgr2=0x7fb9c41298a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:25.046 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.046+0000 7fb9cae72700 1 --2- 172.21.15.40:0/639891357 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9c40b31f0 0x7fb9c41298a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fb9b8002f30 tx=0x7fb9b8015000).stop 2020-11-20T13:56:25.050 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.049+0000 7fb9cae72700 1 -- 172.21.15.40:0/639891357 shutdown_connections 2020-11-20T13:56:25.050 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.049+0000 7fb9cae72700 1 --2- 172.21.15.40:0/639891357 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fb9a4051320 0x7fb9a40537e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:25.050 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.049+0000 7fb9cae72700 1 --2- 172.21.15.40:0/639891357 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb9c40b1450 0x7fb9c412bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:25.051 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.049+0000 7fb9cae72700 1 --2- 172.21.15.40:0/639891357 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9c40b6c80 0x7fb9c4129d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:25.051 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.049+0000 7fb9cae72700 1 --2- 172.21.15.40:0/639891357 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9c40b31f0 0x7fb9c41298a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:25.051 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.049+0000 7fb9cae72700 1 -- 172.21.15.40:0/639891357 >> 172.21.15.40:0/639891357 conn(0x7fb9c40ba1d0 msgr2=0x7fb9c40b40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:25.051 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.050+0000 7fb9cae72700 1 -- 172.21.15.40:0/639891357 shutdown_connections 2020-11-20T13:56:25.051 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.050+0000 7fb9cae72700 1 -- 172.21.15.40:0/639891357 wait complete. 2020-11-20T13:56:25.053 INFO:teuthology.orchestra.run.smithi040.stderr:set pool 4 allow_ec_overwrites to true 2020-11-20T13:56:25.065 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2020-11-20T13:56:25.191 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.190+0000 7fe432e31700 1 Processor -- start 2020-11-20T13:56:25.191 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.190+0000 7fe432e31700 1 -- start start 2020-11-20T13:56:25.193 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.191+0000 7fe432e31700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe42c0b6c80 0x7fe42c0b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:25.193 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.191+0000 7fe432e31700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe42c144bd0 0x7fe42c0b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:25.194 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.191+0000 7fe432e31700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe42c0b0d10 con 0x7fe42c0b45e0 2020-11-20T13:56:25.194 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.191+0000 7fe432e31700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe42c145060 con 0x7fe42c144bd0 2020-11-20T13:56:25.194 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.191+0000 7fe432e31700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe42c0b0fb0 con 0x7fe42c0b6c80 2020-11-20T13:56:25.194 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.191+0000 7fe42bfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe42c144bd0 0x7fe42c0b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:25.195 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.191+0000 7fe42bfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe42c144bd0 0x7fe42c0b40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54460/0 (socket says 172.21.15.40:54460) 2020-11-20T13:56:25.195 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.191+0000 7fe42bfff700 1 -- 172.21.15.40:0/2087173091 learned_addr learned my addr 172.21.15.40:0/2087173091 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:25.195 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.191+0000 7fe430bcd700 1 --2- 172.21.15.40:0/2087173091 >> v2:172.21.15.116:3300/0 conn(0x7fe42c0b6c80 0x7fe42c0b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:25.195 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.191+0000 7fe42bfff700 1 -- 172.21.15.40:0/2087173091 >> v2:172.21.15.116:3300/0 conn(0x7fe42c0b6c80 msgr2=0x7fe42c0b31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:25.195 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.191+0000 7fe42bfff700 1 --2- 172.21.15.40:0/2087173091 >> v2:172.21.15.116:3300/0 conn(0x7fe42c0b6c80 0x7fe42c0b31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:25.196 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.191+0000 7fe42bfff700 1 -- 172.21.15.40:0/2087173091 >> v1:172.21.15.116:6789/0 conn(0x7fe42c0b45e0 legacy=0x7fe42c0b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:25.196 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.192+0000 7fe42bfff700 1 -- 172.21.15.40:0/2087173091 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe42c0b7060 con 0x7fe42c144bd0 2020-11-20T13:56:25.196 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.192+0000 7fe430bcd700 1 --2- 172.21.15.40:0/2087173091 >> v2:172.21.15.116:3300/0 conn(0x7fe42c0b6c80 0x7fe42c0b31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:25.196 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.192+0000 7fe42bfff700 1 --2- 172.21.15.40:0/2087173091 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe42c144bd0 0x7fe42c0b40f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fe420002f30 tx=0x7fe420006e80).ready entity=mon.0 client_cookie=40c0a435cbb820fb server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:25.196 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.192+0000 7fe42b7fe700 1 -- 172.21.15.40:0/2087173091 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe4200097f0 con 0x7fe42c144bd0 2020-11-20T13:56:25.196 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.192+0000 7fe42b7fe700 1 -- 172.21.15.40:0/2087173091 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe420009950 con 0x7fe42c144bd0 2020-11-20T13:56:25.197 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.192+0000 7fe42b7fe700 1 -- 172.21.15.40:0/2087173091 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe420009cb0 con 0x7fe42c144bd0 2020-11-20T13:56:25.197 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.192+0000 7fe432e31700 1 -- 172.21.15.40:0/2087173091 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe42c144bd0 msgr2=0x7fe42c0b40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:25.197 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.192+0000 7fe432e31700 1 --2- 172.21.15.40:0/2087173091 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe42c144bd0 0x7fe42c0b40f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fe420002f30 tx=0x7fe420006e80).stop 2020-11-20T13:56:25.197 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.192+0000 7fe432e31700 1 -- 172.21.15.40:0/2087173091 shutdown_connections 2020-11-20T13:56:25.198 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.192+0000 7fe432e31700 1 --2- 172.21.15.40:0/2087173091 >> v2:172.21.15.116:3300/0 conn(0x7fe42c0b6c80 0x7fe42c0b31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:25.198 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.192+0000 7fe432e31700 1 --2- 172.21.15.40:0/2087173091 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe42c144bd0 0x7fe42c0b40f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:25.198 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.192+0000 7fe432e31700 1 -- 172.21.15.40:0/2087173091 >> 172.21.15.40:0/2087173091 conn(0x7fe42c0ba1d0 msgr2=0x7fe42c0b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:25.198 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.192+0000 7fe432e31700 1 -- 172.21.15.40:0/2087173091 shutdown_connections 2020-11-20T13:56:25.198 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.192+0000 7fe432e31700 1 -- 172.21.15.40:0/2087173091 wait complete. 2020-11-20T13:56:25.198 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.193+0000 7fe432e31700 1 Processor -- start 2020-11-20T13:56:25.199 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.193+0000 7fe432e31700 1 -- start start 2020-11-20T13:56:25.203 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.194+0000 7fe432e31700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe42c0b45e0 0x7fe42c143420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:25.203 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.194+0000 7fe432e31700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe42c0b6c80 0x7fe42c0ae6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:25.203 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.194+0000 7fe432e31700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe42c144bd0 0x7fe42c0aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:25.203 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.194+0000 7fe432e31700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c12b2f0 con 0x7fe42c0b6c80 2020-11-20T13:56:25.203 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.194+0000 7fe432e31700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c0634b0 con 0x7fe42c0b45e0 2020-11-20T13:56:25.204 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.194+0000 7fe432e31700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c063750 con 0x7fe42c144bd0 2020-11-20T13:56:25.204 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.194+0000 7fe42bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe42c0b6c80 0x7fe42c0ae6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:25.204 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.194+0000 7fe42bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe42c0b6c80 0x7fe42c0ae6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54468/0 (socket says 172.21.15.40:54468) 2020-11-20T13:56:25.204 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.194+0000 7fe42bfff700 1 -- 172.21.15.40:0/2986393340 learned_addr learned my addr 172.21.15.40:0/2986393340 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:25.204 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.195+0000 7fe430bcd700 1 --2- 172.21.15.40:0/2986393340 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe42c0b45e0 0x7fe42c143420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:25.205 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.195+0000 7fe4313ce700 1 --2- 172.21.15.40:0/2986393340 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe42c144bd0 0x7fe42c0aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:25.205 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.195+0000 7fe42bfff700 1 -- 172.21.15.40:0/2986393340 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe42c144bd0 msgr2=0x7fe42c0aebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:25.205 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.195+0000 7fe42bfff700 1 --2- 172.21.15.40:0/2986393340 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe42c144bd0 0x7fe42c0aebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:25.205 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.195+0000 7fe42bfff700 1 -- 172.21.15.40:0/2986393340 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe42c0b45e0 msgr2=0x7fe42c143420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:25.205 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.195+0000 7fe42bfff700 1 --2- 172.21.15.40:0/2986393340 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe42c0b45e0 0x7fe42c143420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:25.206 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.195+0000 7fe42bfff700 1 -- 172.21.15.40:0/2986393340 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe42c12afe0 con 0x7fe42c0b6c80 2020-11-20T13:56:25.206 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.195+0000 7fe4313ce700 1 --2- 172.21.15.40:0/2986393340 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe42c144bd0 0x7fe42c0aebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:25.206 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.195+0000 7fe42bfff700 1 --2- 172.21.15.40:0/2986393340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe42c0b6c80 0x7fe42c0ae6f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fe420002f30 tx=0x7fe420015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:25.206 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.195+0000 7fe429ffb700 1 -- 172.21.15.40:0/2986393340 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe420011d70 con 0x7fe42c0b6c80 2020-11-20T13:56:25.206 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.195+0000 7fe429ffb700 1 -- 172.21.15.40:0/2986393340 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe42001e460 con 0x7fe42c0b6c80 2020-11-20T13:56:25.207 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.195+0000 7fe432e31700 1 -- 172.21.15.40:0/2986393340 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe42c127db0 con 0x7fe42c0b6c80 2020-11-20T13:56:25.207 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.195+0000 7fe429ffb700 1 -- 172.21.15.40:0/2986393340 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe42001e7c0 con 0x7fe42c0b6c80 2020-11-20T13:56:25.207 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.195+0000 7fe432e31700 1 -- 172.21.15.40:0/2986393340 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe42c128220 con 0x7fe42c0b6c80 2020-11-20T13:56:25.207 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.196+0000 7fe432e31700 1 -- 172.21.15.40:0/2986393340 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe42c063910 con 0x7fe42c0b6c80 2020-11-20T13:56:25.207 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.196+0000 7fe429ffb700 1 -- 172.21.15.40:0/2986393340 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fe42001a040 con 0x7fe42c0b6c80 2020-11-20T13:56:25.208 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.196+0000 7fe429ffb700 1 --2- 172.21.15.40:0/2986393340 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fe40c051320 0x7fe40c0537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:25.208 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.196+0000 7fe429ffb700 1 -- 172.21.15.40:0/2986393340 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6315+0+0 (secure 0 0 0) 0x7fe42006a050 con 0x7fe42c0b6c80 2020-11-20T13:56:25.208 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.197+0000 7fe430bcd700 1 --2- 172.21.15.40:0/2986393340 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fe40c051320 0x7fe40c0537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:25.208 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.197+0000 7fe430bcd700 1 --2- 172.21.15.40:0/2986393340 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fe40c051320 0x7fe40c0537e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fe41c00c000 tx=0x7fe41c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:25.208 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.200+0000 7fe429ffb700 1 -- 172.21.15.40:0/2986393340 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fe420042290 con 0x7fe42c0b6c80 2020-11-20T13:56:25.427 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.427+0000 7fe432e31700 1 -- 172.21.15.40:0/2986393340 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fe42c057050 con 0x7fe42c0b6c80 2020-11-20T13:56:25.435 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.431+0000 7fe429ffb700 1 -- 172.21.15.40:0/2986393340 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 187+0+0 (secure 0 0 0) 0x7fe4200458b0 con 0x7fe42c0b6c80 2020-11-20T13:56:25.444 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.444+0000 7fe432e31700 1 -- 172.21.15.40:0/2986393340 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fe40c051320 msgr2=0x7fe40c0537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:25.445 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.444+0000 7fe432e31700 1 --2- 172.21.15.40:0/2986393340 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fe40c051320 0x7fe40c0537e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fe41c00c000 tx=0x7fe41c008040).stop 2020-11-20T13:56:25.445 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.444+0000 7fe432e31700 1 -- 172.21.15.40:0/2986393340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe42c0b6c80 msgr2=0x7fe42c0ae6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:25.445 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.444+0000 7fe432e31700 1 --2- 172.21.15.40:0/2986393340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe42c0b6c80 0x7fe42c0ae6f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fe420002f30 tx=0x7fe420015000).stop 2020-11-20T13:56:25.446 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.445+0000 7fe432e31700 1 -- 172.21.15.40:0/2986393340 shutdown_connections 2020-11-20T13:56:25.446 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.445+0000 7fe432e31700 1 --2- 172.21.15.40:0/2986393340 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fe40c051320 0x7fe40c0537e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:25.446 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.445+0000 7fe432e31700 1 --2- 172.21.15.40:0/2986393340 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe42c144bd0 0x7fe42c0aebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:25.447 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.445+0000 7fe432e31700 1 --2- 172.21.15.40:0/2986393340 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe42c0b45e0 0x7fe42c143420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:25.447 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.445+0000 7fe432e31700 1 --2- 172.21.15.40:0/2986393340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe42c0b6c80 0x7fe42c0ae6f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:25.447 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.445+0000 7fe432e31700 1 -- 172.21.15.40:0/2986393340 >> 172.21.15.40:0/2986393340 conn(0x7fe42c0ba1d0 msgr2=0x7fe42c144360 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:25.447 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.445+0000 7fe432e31700 1 -- 172.21.15.40:0/2986393340 shutdown_connections 2020-11-20T13:56:25.447 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.445+0000 7fe432e31700 1 -- 172.21.15.40:0/2986393340 wait complete. 2020-11-20T13:56:25.448 INFO:teuthology.orchestra.run.smithi040.stderr:new fs with metadata pool 3 and data pool 4 2020-11-20T13:56:25.457 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-20T13:56:25.572 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.572+0000 7f8c86948700 1 Processor -- start 2020-11-20T13:56:25.573 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.572+0000 7f8c86948700 1 -- start start 2020-11-20T13:56:25.576 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.573+0000 7f8c86948700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8c80098dc0 0x7f8c800b7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:25.576 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.573+0000 7f8c86948700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c800afd90 con 0x7f8c800b7d80 2020-11-20T13:56:25.577 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.573+0000 7f8c86948700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c800b0030 con 0x7f8c800b6de0 2020-11-20T13:56:25.577 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.573+0000 7f8c86948700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c800b5090 con 0x7f8c80098dc0 2020-11-20T13:56:25.577 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.573+0000 7f8c84ee5700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8c800b7d80 0x7f8c800b81d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45084/0 (socket says 172.21.15.40:45084) 2020-11-20T13:56:25.577 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.573+0000 7f8c84ee5700 1 -- 172.21.15.40:0/847959831 learned_addr learned my addr 172.21.15.40:0/847959831 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:25.577 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.573+0000 7f8c77fff700 1 --2- 172.21.15.40:0/847959831 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8c80098dc0 0x7f8c800b7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:25.578 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.573+0000 7f8c7f7fe700 1 -- 172.21.15.40:0/847959831 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2774798808 0 0) 0x7f8c800afd90 con 0x7f8c800b7d80 2020-11-20T13:56:25.578 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.573+0000 7f8c7f7fe700 1 -- 172.21.15.40:0/847959831 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c60003230 con 0x7f8c800b7d80 2020-11-20T13:56:25.578 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.574+0000 7f8c7f7fe700 1 -- 172.21.15.40:0/847959831 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 525244720 0 0) 0x7f8c60003230 con 0x7f8c800b7d80 2020-11-20T13:56:25.579 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.574+0000 7f8c7f7fe700 1 -- 172.21.15.40:0/847959831 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8c80098dc0 msgr2=0x7f8c800b7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:25.580 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.574+0000 7f8c7f7fe700 1 --2- 172.21.15.40:0/847959831 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8c80098dc0 0x7f8c800b7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:25.580 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.574+0000 7f8c7f7fe700 1 -- 172.21.15.40:0/847959831 >> v1:172.21.15.116:6789/0 conn(0x7f8c800b6de0 legacy=0x7f8c80144bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:25.580 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.574+0000 7f8c7f7fe700 1 -- 172.21.15.40:0/847959831 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c800b71c0 con 0x7f8c800b7d80 2020-11-20T13:56:25.581 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.574+0000 7f8c7f7fe700 1 -- 172.21.15.40:0/847959831 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2366654341 0 0) 0x7f8c78001a50 con 0x7f8c800b7d80 2020-11-20T13:56:25.581 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.574+0000 7f8c7f7fe700 1 -- 172.21.15.40:0/847959831 >> v1:172.21.15.40:6789/0 conn(0x7f8c800b7d80 legacy=0x7f8c800b81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:25.581 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.574+0000 7f8c7f7fe700 1 --2- 172.21.15.40:0/847959831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c60003750 0x7f8c60003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:25.581 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.574+0000 7f8c7f7fe700 1 --2- 172.21.15.40:0/847959831 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c60004090 0x7f8c60004570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:25.581 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.574+0000 7f8c7f7fe700 1 --2- 172.21.15.40:0/847959831 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8c80098dc0 0x7f8c60004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:25.582 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.574+0000 7f8c7f7fe700 1 -- 172.21.15.40:0/847959831 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c800b5090 con 0x7f8c60003750 2020-11-20T13:56:25.582 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.574+0000 7f8c7f7fe700 1 -- 172.21.15.40:0/847959831 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c60005110 con 0x7f8c60004090 2020-11-20T13:56:25.583 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.574+0000 7f8c7f7fe700 1 -- 172.21.15.40:0/847959831 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c600052d0 con 0x7f8c80098dc0 2020-11-20T13:56:25.583 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.574+0000 7f8c7ffff700 1 --2- 172.21.15.40:0/847959831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c60003750 0x7f8c60003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:25.583 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.574+0000 7f8c77fff700 1 --2- 172.21.15.40:0/847959831 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c60004090 0x7f8c60004570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:25.583 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.574+0000 7f8c84ee5700 1 --2- 172.21.15.40:0/847959831 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8c80098dc0 0x7f8c60004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:25.583 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.574+0000 7f8c7ffff700 1 -- 172.21.15.40:0/847959831 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8c80098dc0 msgr2=0x7f8c60004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:25.584 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.574+0000 7f8c7ffff700 1 --2- 172.21.15.40:0/847959831 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8c80098dc0 0x7f8c60004bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:25.584 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.574+0000 7f8c7ffff700 1 -- 172.21.15.40:0/847959831 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c60004090 msgr2=0x7f8c60004570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:25.584 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.574+0000 7f8c7ffff700 1 --2- 172.21.15.40:0/847959831 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c60004090 0x7f8c60004570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:25.584 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.574+0000 7f8c7ffff700 1 -- 172.21.15.40:0/847959831 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c60003230 con 0x7f8c60003750 2020-11-20T13:56:25.584 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.574+0000 7f8c84ee5700 1 --2- 172.21.15.40:0/847959831 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8c80098dc0 0x7f8c60004bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:25.585 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.574+0000 7f8c7ffff700 1 --2- 172.21.15.40:0/847959831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c60003750 0x7f8c60003b30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f8c70000fe0 tx=0x7f8c70007ab0).ready entity=mon.0 client_cookie=354e46caf6db11a8 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:25.585 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.574+0000 7f8c7f7fe700 1 -- 172.21.15.40:0/847959831 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c7000c870 con 0x7f8c60003750 2020-11-20T13:56:25.586 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.574+0000 7f8c7f7fe700 1 -- 172.21.15.40:0/847959831 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c7000c9d0 con 0x7f8c60003750 2020-11-20T13:56:25.586 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.575+0000 7f8c7f7fe700 1 -- 172.21.15.40:0/847959831 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c7000cd30 con 0x7f8c60003750 2020-11-20T13:56:25.586 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.575+0000 7f8c86948700 1 -- 172.21.15.40:0/847959831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c60003750 msgr2=0x7f8c60003b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:25.586 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.575+0000 7f8c86948700 1 --2- 172.21.15.40:0/847959831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c60003750 0x7f8c60003b30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f8c70000fe0 tx=0x7f8c70007ab0).stop 2020-11-20T13:56:25.587 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.575+0000 7f8c84ee5700 1 -- 172.21.15.40:0/847959831 reap_dead start 2020-11-20T13:56:25.587 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.575+0000 7f8c86948700 1 -- 172.21.15.40:0/847959831 shutdown_connections 2020-11-20T13:56:25.587 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.575+0000 7f8c86948700 1 -- 172.21.15.40:0/847959831 >> 172.21.15.40:0/847959831 conn(0x7f8c800ba380 msgr2=0x7f8c800b6350 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:25.587 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.575+0000 7f8c86948700 1 -- 172.21.15.40:0/847959831 shutdown_connections 2020-11-20T13:56:25.587 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.575+0000 7f8c86948700 1 -- 172.21.15.40:0/847959831 wait complete. 2020-11-20T13:56:25.588 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.576+0000 7f8c86948700 1 Processor -- start 2020-11-20T13:56:25.588 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.576+0000 7f8c86948700 1 -- start start 2020-11-20T13:56:25.592 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.577+0000 7f8c86948700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8c800b1e40 0x7f8c80064950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:25.592 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.577+0000 7f8c86948700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c800b2220 0x7f8c80099520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:25.592 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.577+0000 7f8c86948700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c80142e20 0x7f8c80099a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:25.592 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.577+0000 7f8c86948700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c80067580 con 0x7f8c800b2220 2020-11-20T13:56:25.593 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.577+0000 7f8c86948700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c80063f70 con 0x7f8c80142e20 2020-11-20T13:56:25.593 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.577+0000 7f8c86948700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c800643e0 con 0x7f8c800b1e40 2020-11-20T13:56:25.593 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.577+0000 7f8c77fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c800b2220 0x7f8c80099520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:25.593 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.577+0000 7f8c77fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c800b2220 0x7f8c80099520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54498/0 (socket says 172.21.15.40:54498) 2020-11-20T13:56:25.593 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.577+0000 7f8c77fff700 1 -- 172.21.15.40:0/1717966051 learned_addr learned my addr 172.21.15.40:0/1717966051 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:25.594 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.577+0000 7f8c7ffff700 1 --2- 172.21.15.40:0/1717966051 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8c800b1e40 0x7f8c80064950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:25.594 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.577+0000 7f8c84ee5700 1 --2- 172.21.15.40:0/1717966051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c80142e20 0x7f8c80099a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:25.594 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.577+0000 7f8c77fff700 1 -- 172.21.15.40:0/1717966051 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8c800b1e40 msgr2=0x7f8c80064950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:25.594 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.577+0000 7f8c77fff700 1 --2- 172.21.15.40:0/1717966051 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8c800b1e40 0x7f8c80064950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:25.595 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.577+0000 7f8c77fff700 1 -- 172.21.15.40:0/1717966051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c80142e20 msgr2=0x7f8c80099a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:25.595 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.577+0000 7f8c77fff700 1 --2- 172.21.15.40:0/1717966051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c80142e20 0x7f8c80099a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:25.595 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.577+0000 7f8c77fff700 1 -- 172.21.15.40:0/1717966051 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c80067770 con 0x7f8c800b2220 2020-11-20T13:56:25.595 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.577+0000 7f8c84ee5700 1 --2- 172.21.15.40:0/1717966051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c80142e20 0x7f8c80099a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:25.596 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.577+0000 7f8c77fff700 1 --2- 172.21.15.40:0/1717966051 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c800b2220 0x7f8c80099520 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f8c80065610 tx=0x7f8c68008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:25.596 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.577+0000 7f8c7dffb700 1 -- 172.21.15.40:0/1717966051 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c6800f980 con 0x7f8c800b2220 2020-11-20T13:56:25.596 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.577+0000 7f8c7dffb700 1 -- 172.21.15.40:0/1717966051 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c6800fae0 con 0x7f8c800b2220 2020-11-20T13:56:25.596 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.577+0000 7f8c86948700 1 -- 172.21.15.40:0/1717966051 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c80099f00 con 0x7f8c800b2220 2020-11-20T13:56:25.597 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.577+0000 7f8c7dffb700 1 -- 172.21.15.40:0/1717966051 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c6800fc50 con 0x7f8c800b2220 2020-11-20T13:56:25.597 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.577+0000 7f8c86948700 1 -- 172.21.15.40:0/1717966051 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c80142730 con 0x7f8c800b2220 2020-11-20T13:56:25.597 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.578+0000 7f8c86948700 1 -- 172.21.15.40:0/1717966051 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c80064520 con 0x7f8c800b2220 2020-11-20T13:56:25.597 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.578+0000 7f8c7dffb700 1 -- 172.21.15.40:0/1717966051 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f8c68024070 con 0x7f8c800b2220 2020-11-20T13:56:25.597 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.579+0000 7f8c7dffb700 1 --2- 172.21.15.40:0/1717966051 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f8c640513c0 0x7f8c64053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:25.598 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.579+0000 7f8c7dffb700 1 -- 172.21.15.40:0/1717966051 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f8c68069920 con 0x7f8c800b2220 2020-11-20T13:56:25.598 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.579+0000 7f8c7ffff700 1 --2- 172.21.15.40:0/1717966051 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f8c640513c0 0x7f8c64053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:25.598 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.579+0000 7f8c7ffff700 1 --2- 172.21.15.40:0/1717966051 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f8c640513c0 0x7f8c64053880 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f8c7001a000 tx=0x7f8c70016000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:25.598 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.582+0000 7f8c7dffb700 1 -- 172.21.15.40:0/1717966051 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f8c68041b40 con 0x7f8c800b2220 2020-11-20T13:56:25.741 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.741+0000 7f8c86948700 1 -- 172.21.15.40:0/1717966051 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c80057050 con 0x7f8c800b2220 2020-11-20T13:56:25.786 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.742+0000 7f8c7dffb700 1 -- 172.21.15.40:0/1717966051 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16347 (secure 0 0 0) 0x7f8c68045160 con 0x7f8c800b2220 2020-11-20T13:56:25.787 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.743+0000 7f8c86948700 1 -- 172.21.15.40:0/1717966051 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f8c640513c0 msgr2=0x7f8c64053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:25.787 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.743+0000 7f8c86948700 1 --2- 172.21.15.40:0/1717966051 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f8c640513c0 0x7f8c64053880 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f8c7001a000 tx=0x7f8c70016000).stop 2020-11-20T13:56:25.787 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.743+0000 7f8c86948700 1 -- 172.21.15.40:0/1717966051 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c800b2220 msgr2=0x7f8c80099520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:25.787 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.743+0000 7f8c86948700 1 --2- 172.21.15.40:0/1717966051 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c800b2220 0x7f8c80099520 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f8c80065610 tx=0x7f8c68008040).stop 2020-11-20T13:56:25.788 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.743+0000 7f8c86948700 1 -- 172.21.15.40:0/1717966051 shutdown_connections 2020-11-20T13:56:25.788 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.743+0000 7f8c86948700 1 --2- 172.21.15.40:0/1717966051 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f8c640513c0 0x7f8c64053880 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:25.788 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.743+0000 7f8c86948700 1 --2- 172.21.15.40:0/1717966051 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8c800b1e40 0x7f8c80064950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:25.788 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.743+0000 7f8c86948700 1 --2- 172.21.15.40:0/1717966051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c80142e20 0x7f8c80099a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:25.788 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.743+0000 7f8c86948700 1 --2- 172.21.15.40:0/1717966051 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c800b2220 0x7f8c80099520 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:25.789 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.743+0000 7f8c86948700 1 -- 172.21.15.40:0/1717966051 >> 172.21.15.40:0/1717966051 conn(0x7f8c800ba380 msgr2=0x7f8c800ae810 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:25.789 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.743+0000 7f8c86948700 1 -- 172.21.15.40:0/1717966051 shutdown_connections 2020-11-20T13:56:25.789 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.743+0000 7f8c86948700 1 -- 172.21.15.40:0/1717966051 wait complete. 2020-11-20T13:56:25.789 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:56:25.792 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":24,"fsid":"da1ed737-5174-430f-a3c9-56f6bbfad13c","created":"2020-11-20T13:56:04.990651+0000","modified":"2020-11-20T13:56:25.428032+0000","last_up_change":"2020-11-20T13:56:13.893026+0000","last_in_change":"2020-11-20T13:56:13.893026+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T13:56:08.330017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T13:56:17.939684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T13:56:20.563740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T13:56:23.062623+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9ea4c667-8938-46d6-a5ca-092d55c1f79e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":22825},{"type":"v1","addr":"172.21.15.40:6818","nonce":22825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":22825},{"type":"v1","addr":"172.21.15.40:6822","nonce":22825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":22825},{"type":"v1","addr":"172.21.15.40:6830","nonce":22825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":22825},{"type":"v1","addr":"172.21.15.40:6826","nonce":22825}]},"public_addr":"172.21.15.40:6818/22825","cluster_addr":"172.21.15.40:6822/22825","heartbeat_back_addr":"172.21.15.40:6830/22825","heartbeat_front_addr":"172.21.15.40:6826/22825","state":["exists","up"]},{"osd":1,"uuid":"515708d9-5087-4e55-8d14-cdea1c00eacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":22823},{"type":"v1","addr":"172.21.15.40:6801","nonce":22823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":22823},{"type":"v1","addr":"172.21.15.40:6803","nonce":22823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":22823},{"type":"v1","addr":"172.21.15.40:6807","nonce":22823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":22823},{"type":"v1","addr":"172.21.15.40:6805","nonce":22823}]},"public_addr":"172.21.15.40:6801/22823","cluster_addr":"172.21.15.40:6803/22823","heartbeat_back_addr":"172.21.15.40:6807/22823","heartbeat_front_addr":"172.21.15.40:6805/22823","state":["exists","up"]},{"osd":2,"uuid":"4d86bca7-743e-4983-b35f-36676c405b36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":22824},{"type":"v1","addr":"172.21.15.40:6809","nonce":22824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":22824},{"type":"v1","addr":"172.21.15.40:6811","nonce":22824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":22824},{"type":"v1","addr":"172.21.15.40:6815","nonce":22824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":22824},{"type":"v1","addr":"172.21.15.40:6813","nonce":22824}]},"public_addr":"172.21.15.40:6809/22824","cluster_addr":"172.21.15.40:6811/22824","heartbeat_back_addr":"172.21.15.40:6815/22824","heartbeat_front_addr":"172.21.15.40:6813/22824","state":["exists","up"]},{"osd":3,"uuid":"f4dc726b-6b34-4e8b-8eeb-87060b58002d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":22826},{"type":"v1","addr":"172.21.15.40:6819","nonce":22826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":22826},{"type":"v1","addr":"172.21.15.40:6823","nonce":22826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":22826},{"type":"v1","addr":"172.21.15.40:6831","nonce":22826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":22826},{"type":"v1","addr":"172.21.15.40:6827","nonce":22826}]},"public_addr":"172.21.15.40:6819/22826","cluster_addr":"172.21.15.40:6823/22826","heartbeat_back_addr":"172.21.15.40:6831/22826","heartbeat_front_addr":"172.21.15.40:6827/22826","state":["exists","up"]},{"osd":4,"uuid":"25a6d53b-d1f0-4498-b7d5-ffa38661d403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":23181},{"type":"v1","addr":"172.21.15.116:6819","nonce":23181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":23181},{"type":"v1","addr":"172.21.15.116:6821","nonce":23181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":23181},{"type":"v1","addr":"172.21.15.116:6825","nonce":23181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":23181},{"type":"v1","addr":"172.21.15.116:6823","nonce":23181}]},"public_addr":"172.21.15.116:6819/23181","cluster_addr":"172.21.15.116:6821/23181","heartbeat_back_addr":"172.21.15.116:6825/23181","heartbeat_front_addr":"172.21.15.116:6823/23181","state":["exists","up"]},{"osd":5,"uuid":"7d7a5af8-229b-4351-9a21-a3d509cd4635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":23171},{"type":"v1","addr":"172.21.15.116:6803","nonce":23171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":23171},{"type":"v1","addr":"172.21.15.116:6805","nonce":23171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":23171},{"type":"v1","addr":"172.21.15.116:6809","nonce":23171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":23171},{"type":"v1","addr":"172.21.15.116:6807","nonce":23171}]},"public_addr":"172.21.15.116:6803/23171","cluster_addr":"172.21.15.116:6805/23171","heartbeat_back_addr":"172.21.15.116:6809/23171","heartbeat_front_addr":"172.21.15.116:6807/23171","state":["exists","up"]},{"osd":6,"uuid":"8247d248-a140-4e36-9796-9847a97df58d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":23172},{"type":"v1","addr":"172.21.15.116:6811","nonce":23172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":23172},{"type":"v1","addr":"172.21.15.116:6813","nonce":23172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":23172},{"type":"v1","addr":"172.21.15.116:6817","nonce":23172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":23172},{"type":"v1","addr":"172.21.15.116:6815","nonce":23172}]},"public_addr":"172.21.15.116:6811/23172","cluster_addr":"172.21.15.116:6813/23172","heartbeat_back_addr":"172.21.15.116:6817/23172","heartbeat_front_addr":"172.21.15.116:6815/23172","state":["exists","up"]},{"osd":7,"uuid":"472803c4-70b8-499d-b18a-71aaeec6aab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":23196},{"type":"v1","addr":"172.21.15.116:6827","nonce":23196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":23196},{"type":"v1","addr":"172.21.15.116:6829","nonce":23196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":23196},{"type":"v1","addr":"172.21.15.116:6833","nonce":23196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":23196},{"type":"v1","addr":"172.21.15.116:6831","nonce":23196}]},"public_addr":"172.21.15.116:6827/23196","cluster_addr":"172.21.15.116:6829/23196","heartbeat_back_addr":"172.21.15.116:6833/23196","heartbeat_front_addr":"172.21.15.116:6831/23196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.029357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:11.989797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.091097+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.014766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.070860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.071864+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.063400+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.109028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-20T13:56:25.794 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-20T13:56:25.909 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.909+0000 7ff18aa29700 1 Processor -- start 2020-11-20T13:56:25.909 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.909+0000 7ff18aa29700 1 -- start start 2020-11-20T13:56:25.914 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.909+0000 7ff18aa29700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff1840b88c0 0x7ff1840b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:25.914 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.909+0000 7ff18aa29700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff18409b530 0x7ff18409b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:25.914 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.909+0000 7ff18aa29700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1840b6be0 con 0x7ff18409beb0 2020-11-20T13:56:25.914 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.909+0000 7ff18aa29700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff184099000 con 0x7ff18409b530 2020-11-20T13:56:25.915 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.909+0000 7ff18aa29700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1840b4250 con 0x7ff1840b88c0 2020-11-20T13:56:25.915 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.910+0000 7ff1837fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff18409b530 0x7ff18409b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:25.915 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.910+0000 7ff188fc6700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff18409beb0 0x7ff184099270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45100/0 (socket says 172.21.15.40:45100) 2020-11-20T13:56:25.915 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.910+0000 7ff188fc6700 1 -- 172.21.15.40:0/1008978532 learned_addr learned my addr 172.21.15.40:0/1008978532 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:25.915 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.910+0000 7ff183fff700 1 --2- 172.21.15.40:0/1008978532 >> v2:172.21.15.116:3300/0 conn(0x7ff1840b88c0 0x7ff1840b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:25.916 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.910+0000 7ff1837fe700 1 -- 172.21.15.40:0/1008978532 >> v2:172.21.15.116:3300/0 conn(0x7ff1840b88c0 msgr2=0x7ff1840b8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:25.916 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.910+0000 7ff1837fe700 1 --2- 172.21.15.40:0/1008978532 >> v2:172.21.15.116:3300/0 conn(0x7ff1840b88c0 0x7ff1840b8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:25.916 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.910+0000 7ff1837fe700 1 -- 172.21.15.40:0/1008978532 >> v1:172.21.15.40:6789/0 conn(0x7ff18409beb0 legacy=0x7ff184099270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:25.916 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.910+0000 7ff1837fe700 1 -- 172.21.15.40:0/1008978532 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1840b43c0 con 0x7ff18409b530 2020-11-20T13:56:25.917 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.910+0000 7ff1837fe700 1 --2- 172.21.15.40:0/1008978532 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff18409b530 0x7ff18409b9c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7ff174002f30 tx=0x7ff174006dc0).ready entity=mon.0 client_cookie=81a1422dfd7cfbcf server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:25.917 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.910+0000 7ff182ffd700 1 -- 172.21.15.40:0/1008978532 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1740097f0 con 0x7ff18409b530 2020-11-20T13:56:25.918 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.910+0000 7ff182ffd700 1 -- 172.21.15.40:0/1008978532 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff174009950 con 0x7ff18409b530 2020-11-20T13:56:25.918 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.910+0000 7ff182ffd700 1 -- 172.21.15.40:0/1008978532 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff174009cb0 con 0x7ff18409b530 2020-11-20T13:56:25.918 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.910+0000 7ff18aa29700 1 -- 172.21.15.40:0/1008978532 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff18409b530 msgr2=0x7ff18409b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:25.918 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.910+0000 7ff18aa29700 1 --2- 172.21.15.40:0/1008978532 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff18409b530 0x7ff18409b9c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7ff174002f30 tx=0x7ff174006dc0).stop 2020-11-20T13:56:25.919 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.911+0000 7ff18aa29700 1 -- 172.21.15.40:0/1008978532 shutdown_connections 2020-11-20T13:56:25.919 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.911+0000 7ff18aa29700 1 --2- 172.21.15.40:0/1008978532 >> v2:172.21.15.116:3300/0 conn(0x7ff1840b88c0 0x7ff1840b8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:25.919 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.911+0000 7ff18aa29700 1 --2- 172.21.15.40:0/1008978532 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff18409b530 0x7ff18409b9c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:25.919 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.911+0000 7ff18aa29700 1 -- 172.21.15.40:0/1008978532 >> 172.21.15.40:0/1008978532 conn(0x7ff1840ba380 msgr2=0x7ff1840b6350 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:25.920 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.911+0000 7ff18aa29700 1 -- 172.21.15.40:0/1008978532 shutdown_connections 2020-11-20T13:56:25.920 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.911+0000 7ff18aa29700 1 -- 172.21.15.40:0/1008978532 wait complete. 2020-11-20T13:56:25.920 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.912+0000 7ff18aa29700 1 Processor -- start 2020-11-20T13:56:25.920 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.912+0000 7ff18aa29700 1 -- start start 2020-11-20T13:56:25.920 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.912+0000 7ff18aa29700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff18409b530 0x7ff1840b3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:25.921 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.912+0000 7ff18aa29700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff18409beb0 0x7ff184141cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:25.921 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.912+0000 7ff18aa29700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1840b88c0 0x7ff1841421e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:25.921 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.912+0000 7ff18aa29700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1841426d0 con 0x7ff18409b530 2020-11-20T13:56:25.922 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.912+0000 7ff18aa29700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff184148d50 con 0x7ff18409beb0 2020-11-20T13:56:25.922 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.912+0000 7ff18aa29700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff184148810 con 0x7ff1840b88c0 2020-11-20T13:56:25.922 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.913+0000 7ff183fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff18409b530 0x7ff1840b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:25.922 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.913+0000 7ff183fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff18409b530 0x7ff1840b3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54510/0 (socket says 172.21.15.40:54510) 2020-11-20T13:56:25.922 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.913+0000 7ff183fff700 1 -- 172.21.15.40:0/2665856930 learned_addr learned my addr 172.21.15.40:0/2665856930 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:25.923 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.913+0000 7ff188fc6700 1 --2- 172.21.15.40:0/2665856930 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1840b88c0 0x7ff1841421e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:25.923 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.913+0000 7ff1837fe700 1 --2- 172.21.15.40:0/2665856930 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff18409beb0 0x7ff184141cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:25.923 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.913+0000 7ff183fff700 1 -- 172.21.15.40:0/2665856930 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1840b88c0 msgr2=0x7ff1841421e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:25.923 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.913+0000 7ff183fff700 1 --2- 172.21.15.40:0/2665856930 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1840b88c0 0x7ff1841421e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:25.923 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.913+0000 7ff183fff700 1 -- 172.21.15.40:0/2665856930 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff18409beb0 msgr2=0x7ff184141cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:25.924 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.913+0000 7ff183fff700 1 --2- 172.21.15.40:0/2665856930 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff18409beb0 0x7ff184141cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:25.924 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.913+0000 7ff183fff700 1 -- 172.21.15.40:0/2665856930 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff184148ab0 con 0x7ff18409b530 2020-11-20T13:56:25.924 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.913+0000 7ff188fc6700 1 --2- 172.21.15.40:0/2665856930 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1840b88c0 0x7ff1841421e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:25.924 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.913+0000 7ff1837fe700 1 --2- 172.21.15.40:0/2665856930 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff18409beb0 0x7ff184141cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:25.924 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.913+0000 7ff183fff700 1 --2- 172.21.15.40:0/2665856930 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff18409b530 0x7ff1840b3390 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7ff16c003070 tx=0x7ff16c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:25.925 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.913+0000 7ff1817fa700 1 -- 172.21.15.40:0/2665856930 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff16c009980 con 0x7ff18409b530 2020-11-20T13:56:25.925 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.913+0000 7ff1817fa700 1 -- 172.21.15.40:0/2665856930 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff16c009ae0 con 0x7ff18409b530 2020-11-20T13:56:25.925 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.913+0000 7ff18aa29700 1 -- 172.21.15.40:0/2665856930 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1841484c0 con 0x7ff18409b530 2020-11-20T13:56:25.925 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.913+0000 7ff1817fa700 1 -- 172.21.15.40:0/2665856930 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff16c009e40 con 0x7ff18409b530 2020-11-20T13:56:25.925 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.913+0000 7ff18aa29700 1 -- 172.21.15.40:0/2665856930 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff184147570 con 0x7ff18409b530 2020-11-20T13:56:25.926 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.914+0000 7ff18aa29700 1 -- 172.21.15.40:0/2665856930 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff184057050 con 0x7ff18409b530 2020-11-20T13:56:25.927 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.914+0000 7ff1817fa700 1 -- 172.21.15.40:0/2665856930 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7ff16c013480 con 0x7ff18409b530 2020-11-20T13:56:25.927 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.915+0000 7ff1817fa700 1 --2- 172.21.15.40:0/2665856930 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7ff168051210 0x7ff1680536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:25.927 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.915+0000 7ff1817fa700 1 -- 172.21.15.40:0/2665856930 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7ff16c0698b0 con 0x7ff18409b530 2020-11-20T13:56:25.927 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.915+0000 7ff1837fe700 1 --2- 172.21.15.40:0/2665856930 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7ff168051210 0x7ff1680536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:25.928 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.915+0000 7ff1837fe700 1 --2- 172.21.15.40:0/2665856930 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7ff168051210 0x7ff1680536d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7ff174019000 tx=0x7ff174015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:25.928 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:25.918+0000 7ff1817fa700 1 -- 172.21.15.40:0/2665856930 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7ff16c041a70 con 0x7ff18409b530 2020-11-20T13:56:26.081 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.081+0000 7ff18aa29700 1 -- 172.21.15.40:0/2665856930 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff184147350 con 0x7ff18409b530 2020-11-20T13:56:26.081 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.081+0000 7ff1817fa700 1 -- 172.21.15.40:0/2665856930 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16347 (secure 0 0 0) 0x7ff16c045090 con 0x7ff18409b530 2020-11-20T13:56:26.082 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:56:26.082 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":24,"fsid":"da1ed737-5174-430f-a3c9-56f6bbfad13c","created":"2020-11-20T13:56:04.990651+0000","modified":"2020-11-20T13:56:25.428032+0000","last_up_change":"2020-11-20T13:56:13.893026+0000","last_in_change":"2020-11-20T13:56:13.893026+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T13:56:08.330017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T13:56:17.939684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T13:56:20.563740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T13:56:23.062623+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9ea4c667-8938-46d6-a5ca-092d55c1f79e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":22825},{"type":"v1","addr":"172.21.15.40:6818","nonce":22825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":22825},{"type":"v1","addr":"172.21.15.40:6822","nonce":22825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":22825},{"type":"v1","addr":"172.21.15.40:6830","nonce":22825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":22825},{"type":"v1","addr":"172.21.15.40:6826","nonce":22825}]},"public_addr":"172.21.15.40:6818/22825","cluster_addr":"172.21.15.40:6822/22825","heartbeat_back_addr":"172.21.15.40:6830/22825","heartbeat_front_addr":"172.21.15.40:6826/22825","state":["exists","up"]},{"osd":1,"uuid":"515708d9-5087-4e55-8d14-cdea1c00eacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":22823},{"type":"v1","addr":"172.21.15.40:6801","nonce":22823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":22823},{"type":"v1","addr":"172.21.15.40:6803","nonce":22823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":22823},{"type":"v1","addr":"172.21.15.40:6807","nonce":22823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":22823},{"type":"v1","addr":"172.21.15.40:6805","nonce":22823}]},"public_addr":"172.21.15.40:6801/22823","cluster_addr":"172.21.15.40:6803/22823","heartbeat_back_addr":"172.21.15.40:6807/22823","heartbeat_front_addr":"172.21.15.40:6805/22823","state":["exists","up"]},{"osd":2,"uuid":"4d86bca7-743e-4983-b35f-36676c405b36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":22824},{"type":"v1","addr":"172.21.15.40:6809","nonce":22824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":22824},{"type":"v1","addr":"172.21.15.40:6811","nonce":22824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":22824},{"type":"v1","addr":"172.21.15.40:6815","nonce":22824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":22824},{"type":"v1","addr":"172.21.15.40:6813","nonce":22824}]},"public_addr":"172.21.15.40:6809/22824","cluster_addr":"172.21.15.40:6811/22824","heartbeat_back_addr":"172.21.15.40:6815/22824","heartbeat_front_addr":"172.21.15.40:6813/22824","state":["exists","up"]},{"osd":3,"uuid":"f4dc726b-6b34-4e8b-8eeb-87060b58002d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":22826},{"type":"v1","addr":"172.21.15.40:6819","nonce":22826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":22826},{"type":"v1","addr":"172.21.15.40:6823","nonce":22826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":22826},{"type":"v1","addr":"172.21.15.40:6831","nonce":22826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":22826},{"type":"v1","addr":"172.21.15.40:6827","nonce":22826}]},"public_addr":"172.21.15.40:6819/22826","cluster_addr":"172.21.15.40:6823/22826","heartbeat_back_addr":"172.21.15.40:6831/22826","heartbeat_front_addr":"172.21.15.40:6827/22826","state":["exists","up"]},{"osd":4,"uuid":"25a6d53b-d1f0-4498-b7d5-ffa38661d403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":23181},{"type":"v1","addr":"172.21.15.116:6819","nonce":23181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":23181},{"type":"v1","addr":"172.21.15.116:6821","nonce":23181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":23181},{"type":"v1","addr":"172.21.15.116:6825","nonce":23181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":23181},{"type":"v1","addr":"172.21.15.116:6823","nonce":23181}]},"public_addr":"172.21.15.116:6819/23181","cluster_addr":"172.21.15.116:6821/23181","heartbeat_back_addr":"172.21.15.116:6825/23181","heartbeat_front_addr":"172.21.15.116:6823/23181","state":["exists","up"]},{"osd":5,"uuid":"7d7a5af8-229b-4351-9a21-a3d509cd4635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":23171},{"type":"v1","addr":"172.21.15.116:6803","nonce":23171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":23171},{"type":"v1","addr":"172.21.15.116:6805","nonce":23171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":23171},{"type":"v1","addr":"172.21.15.116:6809","nonce":23171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":23171},{"type":"v1","addr":"172.21.15.116:6807","nonce":23171}]},"public_addr":"172.21.15.116:6803/23171","cluster_addr":"172.21.15.116:6805/23171","heartbeat_back_addr":"172.21.15.116:6809/23171","heartbeat_front_addr":"172.21.15.116:6807/23171","state":["exists","up"]},{"osd":6,"uuid":"8247d248-a140-4e36-9796-9847a97df58d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":23172},{"type":"v1","addr":"172.21.15.116:6811","nonce":23172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":23172},{"type":"v1","addr":"172.21.15.116:6813","nonce":23172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":23172},{"type":"v1","addr":"172.21.15.116:6817","nonce":23172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":23172},{"type":"v1","addr":"172.21.15.116:6815","nonce":23172}]},"public_addr":"172.21.15.116:6811/23172","cluster_addr":"172.21.15.116:6813/23172","heartbeat_back_addr":"172.21.15.116:6817/23172","heartbeat_front_addr":"172.21.15.116:6815/23172","state":["exists","up"]},{"osd":7,"uuid":"472803c4-70b8-499d-b18a-71aaeec6aab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":23196},{"type":"v1","addr":"172.21.15.116:6827","nonce":23196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":23196},{"type":"v1","addr":"172.21.15.116:6829","nonce":23196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":23196},{"type":"v1","addr":"172.21.15.116:6833","nonce":23196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":23196},{"type":"v1","addr":"172.21.15.116:6831","nonce":23196}]},"public_addr":"172.21.15.116:6827/23196","cluster_addr":"172.21.15.116:6829/23196","heartbeat_back_addr":"172.21.15.116:6833/23196","heartbeat_front_addr":"172.21.15.116:6831/23196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.029357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:11.989797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.091097+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.014766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.070860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.071864+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.063400+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.109028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-20T13:56:26.084 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.083+0000 7ff18aa29700 1 -- 172.21.15.40:0/2665856930 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7ff168051210 msgr2=0x7ff1680536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:26.084 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.083+0000 7ff18aa29700 1 --2- 172.21.15.40:0/2665856930 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7ff168051210 0x7ff1680536d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7ff174019000 tx=0x7ff174015000).stop 2020-11-20T13:56:26.085 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.083+0000 7ff18aa29700 1 -- 172.21.15.40:0/2665856930 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff18409b530 msgr2=0x7ff1840b3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:26.086 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.083+0000 7ff18aa29700 1 --2- 172.21.15.40:0/2665856930 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff18409b530 0x7ff1840b3390 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7ff16c003070 tx=0x7ff16c008040).stop 2020-11-20T13:56:26.086 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.083+0000 7ff18aa29700 1 -- 172.21.15.40:0/2665856930 shutdown_connections 2020-11-20T13:56:26.086 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.083+0000 7ff18aa29700 1 --2- 172.21.15.40:0/2665856930 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7ff168051210 0x7ff1680536d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:26.086 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.083+0000 7ff18aa29700 1 --2- 172.21.15.40:0/2665856930 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1840b88c0 0x7ff1841421e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.083+0000 7ff18aa29700 1 --2- 172.21.15.40:0/2665856930 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff18409b530 0x7ff1840b3390 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.083+0000 7ff18aa29700 1 --2- 172.21.15.40:0/2665856930 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff18409beb0 0x7ff184141cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.083+0000 7ff18aa29700 1 -- 172.21.15.40:0/2665856930 >> 172.21.15.40:0/2665856930 conn(0x7ff1840ba380 msgr2=0x7ff184142b70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.083+0000 7ff18aa29700 1 -- 172.21.15.40:0/2665856930 shutdown_connections 2020-11-20T13:56:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.083+0000 7ff18aa29700 1 -- 172.21.15.40:0/2665856930 wait complete. 2020-11-20T13:56:26.094 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2020-11-20T13:56:26.211 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.211+0000 7f8506d0b700 1 Processor -- start 2020-11-20T13:56:26.211 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.211+0000 7f8506d0b700 1 -- start start 2020-11-20T13:56:26.218 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.212+0000 7f8506d0b700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f850013f400 0x7f85000b3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:26.218 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.212+0000 7f8506d0b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f850005fa50 0x7f8500149120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:26.219 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.212+0000 7f8506d0b700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85001489e0 con 0x7f85000b6900 2020-11-20T13:56:26.219 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.212+0000 7f8506d0b700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8500148c80 con 0x7f850005fa50 2020-11-20T13:56:26.219 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.212+0000 7f8506d0b700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f850005f5c0 con 0x7f850013f400 2020-11-20T13:56:26.219 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.212+0000 7f84fffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f850005fa50 0x7f8500149120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:26.220 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.212+0000 7f85052a8700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f85000b6900 0x7f8500145290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45118/0 (socket says 172.21.15.40:45118) 2020-11-20T13:56:26.220 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.212+0000 7f85052a8700 1 -- 172.21.15.40:0/875571125 learned_addr learned my addr 172.21.15.40:0/875571125 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:26.220 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.212+0000 7f8504aa7700 1 --2- 172.21.15.40:0/875571125 >> v2:172.21.15.116:3300/0 conn(0x7f850013f400 0x7f85000b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:26.220 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.212+0000 7f84fffff700 1 -- 172.21.15.40:0/875571125 >> v2:172.21.15.116:3300/0 conn(0x7f850013f400 msgr2=0x7f85000b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:26.221 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.212+0000 7f84fffff700 1 --2- 172.21.15.40:0/875571125 >> v2:172.21.15.116:3300/0 conn(0x7f850013f400 0x7f85000b3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:26.221 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.212+0000 7f84fffff700 1 -- 172.21.15.40:0/875571125 >> v1:172.21.15.40:6789/0 conn(0x7f85000b6900 legacy=0x7f8500145290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:26.221 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.212+0000 7f84ff7fe700 1 -- 172.21.15.40:0/875571125 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 946953149 0 0) 0x7f85001489e0 con 0x7f85000b6900 2020-11-20T13:56:26.221 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.212+0000 7f84fffff700 1 -- 172.21.15.40:0/875571125 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8500149610 con 0x7f850005fa50 2020-11-20T13:56:26.222 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.212+0000 7f84fffff700 1 --2- 172.21.15.40:0/875571125 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f850005fa50 0x7f8500149120 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f84f4002f30 tx=0x7f84f4006dc0).ready entity=mon.0 client_cookie=d760a28e3521561b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:26.222 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.212+0000 7f84ff7fe700 1 -- 172.21.15.40:0/875571125 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84f40097b0 con 0x7f850005fa50 2020-11-20T13:56:26.222 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.212+0000 7f84ff7fe700 1 -- 172.21.15.40:0/875571125 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f4009910 con 0x7f850005fa50 2020-11-20T13:56:26.222 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.212+0000 7f84ff7fe700 1 -- 172.21.15.40:0/875571125 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84f4009c70 con 0x7f850005fa50 2020-11-20T13:56:26.222 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.212+0000 7f8506d0b700 1 -- 172.21.15.40:0/875571125 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f850005fa50 msgr2=0x7f8500149120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:26.223 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.212+0000 7f8506d0b700 1 --2- 172.21.15.40:0/875571125 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f850005fa50 0x7f8500149120 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f84f4002f30 tx=0x7f84f4006dc0).stop 2020-11-20T13:56:26.223 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.213+0000 7f8506d0b700 1 -- 172.21.15.40:0/875571125 shutdown_connections 2020-11-20T13:56:26.223 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.213+0000 7f8506d0b700 1 --2- 172.21.15.40:0/875571125 >> v2:172.21.15.116:3300/0 conn(0x7f850013f400 0x7f85000b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:26.223 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.213+0000 7f8506d0b700 1 --2- 172.21.15.40:0/875571125 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f850005fa50 0x7f8500149120 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:26.223 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.213+0000 7f8506d0b700 1 -- 172.21.15.40:0/875571125 >> 172.21.15.40:0/875571125 conn(0x7f85000ba1d0 msgr2=0x7f85000b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:26.224 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.213+0000 7f8506d0b700 1 -- 172.21.15.40:0/875571125 shutdown_connections 2020-11-20T13:56:26.224 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.213+0000 7f8506d0b700 1 -- 172.21.15.40:0/875571125 wait complete. 2020-11-20T13:56:26.224 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.214+0000 7f8506d0b700 1 Processor -- start 2020-11-20T13:56:26.224 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.214+0000 7f8506d0b700 1 -- start start 2020-11-20T13:56:26.224 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.215+0000 7f8506d0b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f850005fa50 0x7f8500141d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:26.225 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.215+0000 7f8506d0b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85000b6900 0x7f8500062870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:26.225 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.215+0000 7f8506d0b700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f850013f400 0x7f8500062d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:26.225 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.215+0000 7f8506d0b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8500155190 con 0x7f85000b6900 2020-11-20T13:56:26.225 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.215+0000 7f8506d0b700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85001436b0 con 0x7f850005fa50 2020-11-20T13:56:26.225 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.215+0000 7f8506d0b700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8500143950 con 0x7f850013f400 2020-11-20T13:56:26.226 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.215+0000 7f84fffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85000b6900 0x7f8500062870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:26.226 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.215+0000 7f84fffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85000b6900 0x7f8500062870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54526/0 (socket says 172.21.15.40:54526) 2020-11-20T13:56:26.226 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.215+0000 7f84fffff700 1 -- 172.21.15.40:0/1596604167 learned_addr learned my addr 172.21.15.40:0/1596604167 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:26.226 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.215+0000 7f85052a8700 1 --2- 172.21.15.40:0/1596604167 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f850013f400 0x7f8500062d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:26.227 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.215+0000 7f84fffff700 1 -- 172.21.15.40:0/1596604167 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f850013f400 msgr2=0x7f8500062d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:26.227 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.215+0000 7f84fffff700 1 --2- 172.21.15.40:0/1596604167 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f850013f400 0x7f8500062d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:26.228 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.215+0000 7f84fffff700 1 -- 172.21.15.40:0/1596604167 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f850005fa50 msgr2=0x7f8500141d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:26.228 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.215+0000 7f8504aa7700 1 --2- 172.21.15.40:0/1596604167 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f850005fa50 0x7f8500141d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:26.228 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.215+0000 7f84fffff700 1 --2- 172.21.15.40:0/1596604167 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f850005fa50 0x7f8500141d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:26.229 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.215+0000 7f84fffff700 1 -- 172.21.15.40:0/1596604167 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85001485e0 con 0x7f85000b6900 2020-11-20T13:56:26.230 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.215+0000 7f85052a8700 1 --2- 172.21.15.40:0/1596604167 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f850013f400 0x7f8500062d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:26.230 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.215+0000 7f84fffff700 1 --2- 172.21.15.40:0/1596604167 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85000b6900 0x7f8500062870 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f84f4002f30 tx=0x7f84f4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:26.231 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.215+0000 7f84fdffb700 1 -- 172.21.15.40:0/1596604167 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84f4006a70 con 0x7f85000b6900 2020-11-20T13:56:26.231 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.216+0000 7f84fdffb700 1 -- 172.21.15.40:0/1596604167 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f4011ca0 con 0x7f85000b6900 2020-11-20T13:56:26.231 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.216+0000 7f84fdffb700 1 -- 172.21.15.40:0/1596604167 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84f401d440 con 0x7f85000b6900 2020-11-20T13:56:26.231 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.216+0000 7f8506d0b700 1 -- 172.21.15.40:0/1596604167 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8500147c90 con 0x7f85000b6900 2020-11-20T13:56:26.231 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.216+0000 7f8506d0b700 1 -- 172.21.15.40:0/1596604167 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8500154e50 con 0x7f85000b6900 2020-11-20T13:56:26.232 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.216+0000 7f8506d0b700 1 -- 172.21.15.40:0/1596604167 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8500147fa0 con 0x7f85000b6900 2020-11-20T13:56:26.232 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.217+0000 7f84fdffb700 1 -- 172.21.15.40:0/1596604167 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f84f4017070 con 0x7f85000b6900 2020-11-20T13:56:26.232 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.217+0000 7f84fdffb700 1 --2- 172.21.15.40:0/1596604167 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f84e0051350 0x7f84e0053810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:26.232 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.217+0000 7f84fdffb700 1 -- 172.21.15.40:0/1596604167 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f84f4069ba0 con 0x7f85000b6900 2020-11-20T13:56:26.233 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.217+0000 7f8504aa7700 1 --2- 172.21.15.40:0/1596604167 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f84e0051350 0x7f84e0053810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:26.233 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.217+0000 7f8504aa7700 1 --2- 172.21.15.40:0/1596604167 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f84e0051350 0x7f84e0053810 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f84f800b000 tx=0x7f84f8008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:26.233 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.220+0000 7f84fdffb700 1 -- 172.21.15.40:0/1596604167 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f8500147fa0 con 0x7f85000b6900 2020-11-20T13:56:26.436 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:26.436+0000 7f8506d0b700 1 -- 172.21.15.40:0/1596604167 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8500147fa0 con 0x7f85000b6900 2020-11-20T13:56:27.435 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.435+0000 7f84fdffb700 1 -- 172.21.15.40:0/1596604167 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8500147fa0 con 0x7f85000b6900 2020-11-20T13:56:27.437 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.437+0000 7f8506d0b700 1 -- 172.21.15.40:0/1596604167 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f84e0051350 msgr2=0x7f84e0053810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:27.438 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.437+0000 7f8506d0b700 1 --2- 172.21.15.40:0/1596604167 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f84e0051350 0x7f84e0053810 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f84f800b000 tx=0x7f84f8008040).stop 2020-11-20T13:56:27.438 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.437+0000 7f8506d0b700 1 -- 172.21.15.40:0/1596604167 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85000b6900 msgr2=0x7f8500062870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:27.438 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.437+0000 7f8506d0b700 1 --2- 172.21.15.40:0/1596604167 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85000b6900 0x7f8500062870 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f84f4002f30 tx=0x7f84f4015000).stop 2020-11-20T13:56:27.438 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.437+0000 7f8506d0b700 1 -- 172.21.15.40:0/1596604167 shutdown_connections 2020-11-20T13:56:27.438 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.437+0000 7f8506d0b700 1 --2- 172.21.15.40:0/1596604167 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f84e0051350 0x7f84e0053810 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:27.439 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.437+0000 7f8506d0b700 1 --2- 172.21.15.40:0/1596604167 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f850013f400 0x7f8500062d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:27.439 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.437+0000 7f8506d0b700 1 --2- 172.21.15.40:0/1596604167 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f850005fa50 0x7f8500141d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:27.439 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.437+0000 7f8506d0b700 1 --2- 172.21.15.40:0/1596604167 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85000b6900 0x7f8500062870 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:27.439 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.437+0000 7f8506d0b700 1 -- 172.21.15.40:0/1596604167 >> 172.21.15.40:0/1596604167 conn(0x7f85000ba1d0 msgr2=0x7f85000b7650 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:27.439 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.437+0000 7f8506d0b700 1 -- 172.21.15.40:0/1596604167 shutdown_connections 2020-11-20T13:56:27.440 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.437+0000 7f8506d0b700 1 -- 172.21.15.40:0/1596604167 wait complete. 2020-11-20T13:56:27.450 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-20T13:56:27.571 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.570+0000 7ff6d5188700 1 Processor -- start 2020-11-20T13:56:27.571 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.570+0000 7ff6d5188700 1 -- start start 2020-11-20T13:56:27.576 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.571+0000 7ff6d5188700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff6d00b88c0 0x7ff6d00b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:27.576 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.571+0000 7ff6d5188700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6d00b6be0 con 0x7ff6d009b530 2020-11-20T13:56:27.576 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.571+0000 7ff6d5188700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6d00b4250 con 0x7ff6d009a280 2020-11-20T13:56:27.577 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.571+0000 7ff6d5188700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d0099000 con 0x7ff6d00b88c0 2020-11-20T13:56:27.577 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.571+0000 7ff6ce59c700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff6d009b530 0x7ff6d009b990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45128/0 (socket says 172.21.15.40:45128) 2020-11-20T13:56:27.577 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.571+0000 7ff6ce59c700 1 -- 172.21.15.40:0/4217621321 learned_addr learned my addr 172.21.15.40:0/4217621321 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:27.577 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.571+0000 7ff6ced9d700 1 --2- 172.21.15.40:0/4217621321 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff6d00b88c0 0x7ff6d00b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:27.577 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.572+0000 7ff6cdd9b700 1 -- 172.21.15.40:0/4217621321 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4219128553 0 0) 0x7ff6d00b6be0 con 0x7ff6d009b530 2020-11-20T13:56:27.578 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.572+0000 7ff6cdd9b700 1 -- 172.21.15.40:0/4217621321 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6b0003230 con 0x7ff6d009b530 2020-11-20T13:56:27.578 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.572+0000 7ff6cdd9b700 1 -- 172.21.15.40:0/4217621321 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2064014415 0 0) 0x7ff6b0003230 con 0x7ff6d009b530 2020-11-20T13:56:27.578 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.572+0000 7ff6cdd9b700 1 -- 172.21.15.40:0/4217621321 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff6d00b88c0 msgr2=0x7ff6d00b8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:27.578 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.572+0000 7ff6cdd9b700 1 --2- 172.21.15.40:0/4217621321 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff6d00b88c0 0x7ff6d00b8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:27.578 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.572+0000 7ff6cdd9b700 1 -- 172.21.15.40:0/4217621321 >> v1:172.21.15.116:6789/0 conn(0x7ff6d009a280 legacy=0x7ff6d014e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:27.579 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.572+0000 7ff6cdd9b700 1 -- 172.21.15.40:0/4217621321 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d00af250 con 0x7ff6d009b530 2020-11-20T13:56:27.579 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.572+0000 7ff6cdd9b700 1 -- 172.21.15.40:0/4217621321 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2366654341 0 0) 0x7ff6b80016d0 con 0x7ff6d009b530 2020-11-20T13:56:27.579 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.572+0000 7ff6cdd9b700 1 -- 172.21.15.40:0/4217621321 >> v1:172.21.15.40:6789/0 conn(0x7ff6d009b530 legacy=0x7ff6d009b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:27.579 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.572+0000 7ff6cdd9b700 1 --2- 172.21.15.40:0/4217621321 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6b0003750 0x7ff6b0003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:27.579 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.572+0000 7ff6cdd9b700 1 --2- 172.21.15.40:0/4217621321 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6b00044d0 0x7ff6b0004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:27.580 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.572+0000 7ff6cdd9b700 1 --2- 172.21.15.40:0/4217621321 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff6d00b88c0 0x7ff6b0004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:27.580 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.572+0000 7ff6cdd9b700 1 -- 172.21.15.40:0/4217621321 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d0099000 con 0x7ff6b00044d0 2020-11-20T13:56:27.580 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.572+0000 7ff6cdd9b700 1 -- 172.21.15.40:0/4217621321 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b0005380 con 0x7ff6b0003750 2020-11-20T13:56:27.581 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.572+0000 7ff6cdd9b700 1 -- 172.21.15.40:0/4217621321 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b0004310 con 0x7ff6d00b88c0 2020-11-20T13:56:27.581 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.572+0000 7ff6ce59c700 1 --2- 172.21.15.40:0/4217621321 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6b00044d0 0x7ff6b0004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:27.581 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.572+0000 7ff6ced9d700 1 --2- 172.21.15.40:0/4217621321 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6b0003750 0x7ff6b0003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:27.581 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.572+0000 7ff6cf59e700 1 --2- 172.21.15.40:0/4217621321 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff6d00b88c0 0x7ff6b0004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:27.582 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.572+0000 7ff6ce59c700 1 -- 172.21.15.40:0/4217621321 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff6d00b88c0 msgr2=0x7ff6b0004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:27.582 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.572+0000 7ff6ce59c700 1 --2- 172.21.15.40:0/4217621321 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff6d00b88c0 0x7ff6b0004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:27.582 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.572+0000 7ff6ce59c700 1 -- 172.21.15.40:0/4217621321 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6b0003750 msgr2=0x7ff6b0003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:27.582 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.572+0000 7ff6ce59c700 1 --2- 172.21.15.40:0/4217621321 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6b0003750 0x7ff6b0003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:27.582 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.572+0000 7ff6ce59c700 1 -- 172.21.15.40:0/4217621321 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b0003230 con 0x7ff6b00044d0 2020-11-20T13:56:27.583 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.572+0000 7ff6ce59c700 1 --2- 172.21.15.40:0/4217621321 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6b00044d0 0x7ff6b0004950 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7ff6b80013e0 tx=0x7ff6b8007740).ready entity=mon.0 client_cookie=8d09798f9df3ee31 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:27.583 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.573+0000 7ff6cdd9b700 1 -- 172.21.15.40:0/4217621321 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6b80070d0 con 0x7ff6b00044d0 2020-11-20T13:56:27.583 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.573+0000 7ff6cdd9b700 1 -- 172.21.15.40:0/4217621321 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6b8007230 con 0x7ff6b00044d0 2020-11-20T13:56:27.583 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.573+0000 7ff6cdd9b700 1 -- 172.21.15.40:0/4217621321 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6b80055e0 con 0x7ff6b00044d0 2020-11-20T13:56:27.584 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.573+0000 7ff6d5188700 1 -- 172.21.15.40:0/4217621321 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6b00044d0 msgr2=0x7ff6b0004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:27.584 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.573+0000 7ff6d5188700 1 --2- 172.21.15.40:0/4217621321 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6b00044d0 0x7ff6b0004950 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7ff6b80013e0 tx=0x7ff6b8007740).stop 2020-11-20T13:56:27.584 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.573+0000 7ff6cf59e700 1 -- 172.21.15.40:0/4217621321 reap_dead start 2020-11-20T13:56:27.585 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.573+0000 7ff6d5188700 1 -- 172.21.15.40:0/4217621321 shutdown_connections 2020-11-20T13:56:27.585 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.573+0000 7ff6d5188700 1 -- 172.21.15.40:0/4217621321 >> 172.21.15.40:0/4217621321 conn(0x7ff6d00ba380 msgr2=0x7ff6d00b6350 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:27.585 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.573+0000 7ff6d5188700 1 -- 172.21.15.40:0/4217621321 shutdown_connections 2020-11-20T13:56:27.586 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.573+0000 7ff6d5188700 1 -- 172.21.15.40:0/4217621321 wait complete. 2020-11-20T13:56:27.586 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.574+0000 7ff6d5188700 1 Processor -- start 2020-11-20T13:56:27.586 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.574+0000 7ff6d5188700 1 -- start start 2020-11-20T13:56:27.587 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.575+0000 7ff6d5188700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6d00b5790 0x7ff6d0142560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:27.587 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.575+0000 7ff6d5188700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6d0154900 0x7ff6d0154d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:27.587 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.575+0000 7ff6d5188700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff6d00b77b0 0x7ff6d00b7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:27.587 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.575+0000 7ff6d5188700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d0145460 con 0x7ff6d0154900 2020-11-20T13:56:27.587 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.575+0000 7ff6d5188700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d0145700 con 0x7ff6d00b5790 2020-11-20T13:56:27.588 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.575+0000 7ff6d5188700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d00b5b70 con 0x7ff6d00b77b0 2020-11-20T13:56:27.588 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.575+0000 7ff6ce59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6d0154900 0x7ff6d0154d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:27.588 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.575+0000 7ff6ce59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6d0154900 0x7ff6d0154d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54542/0 (socket says 172.21.15.40:54542) 2020-11-20T13:56:27.588 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.575+0000 7ff6ce59c700 1 -- 172.21.15.40:0/2278826988 learned_addr learned my addr 172.21.15.40:0/2278826988 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:27.588 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.575+0000 7ff6ced9d700 1 --2- 172.21.15.40:0/2278826988 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6d00b5790 0x7ff6d0142560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:27.589 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.575+0000 7ff6cf59e700 1 --2- 172.21.15.40:0/2278826988 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff6d00b77b0 0x7ff6d00b7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:27.589 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.575+0000 7ff6ce59c700 1 -- 172.21.15.40:0/2278826988 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff6d00b77b0 msgr2=0x7ff6d00b7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:27.589 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.575+0000 7ff6ce59c700 1 --2- 172.21.15.40:0/2278826988 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff6d00b77b0 0x7ff6d00b7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:27.589 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.575+0000 7ff6ce59c700 1 -- 172.21.15.40:0/2278826988 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6d00b5790 msgr2=0x7ff6d0142560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:27.590 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.575+0000 7ff6ce59c700 1 --2- 172.21.15.40:0/2278826988 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6d00b5790 0x7ff6d0142560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:27.590 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.575+0000 7ff6ce59c700 1 -- 172.21.15.40:0/2278826988 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d01543b0 con 0x7ff6d0154900 2020-11-20T13:56:27.590 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.575+0000 7ff6ce59c700 1 --2- 172.21.15.40:0/2278826988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6d0154900 0x7ff6d0154d80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7ff6b80013e0 tx=0x7ff6b8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:27.590 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.575+0000 7ff6c7fff700 1 -- 172.21.15.40:0/2278826988 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6b80076b0 con 0x7ff6d0154900 2020-11-20T13:56:27.591 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.576+0000 7ff6c7fff700 1 -- 172.21.15.40:0/2278826988 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6b8007450 con 0x7ff6d0154900 2020-11-20T13:56:27.591 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.576+0000 7ff6d5188700 1 -- 172.21.15.40:0/2278826988 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6d0145460 con 0x7ff6d0154900 2020-11-20T13:56:27.591 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.576+0000 7ff6c7fff700 1 -- 172.21.15.40:0/2278826988 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6b8020670 con 0x7ff6d0154900 2020-11-20T13:56:27.592 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.576+0000 7ff6d5188700 1 -- 172.21.15.40:0/2278826988 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d0144aa0 con 0x7ff6d0154900 2020-11-20T13:56:27.593 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.576+0000 7ff6d5188700 1 -- 172.21.15.40:0/2278826988 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6d0057050 con 0x7ff6d0154900 2020-11-20T13:56:27.593 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.577+0000 7ff6c7fff700 1 -- 172.21.15.40:0/2278826988 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7ff6b8010070 con 0x7ff6d0154900 2020-11-20T13:56:27.593 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.577+0000 7ff6c7fff700 1 --2- 172.21.15.40:0/2278826988 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7ff6b40513c0 0x7ff6b4053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:27.593 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.577+0000 7ff6c7fff700 1 -- 172.21.15.40:0/2278826988 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7ff6b806a100 con 0x7ff6d0154900 2020-11-20T13:56:27.594 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.577+0000 7ff6ced9d700 1 --2- 172.21.15.40:0/2278826988 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7ff6b40513c0 0x7ff6b4053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:27.594 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.577+0000 7ff6ced9d700 1 --2- 172.21.15.40:0/2278826988 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7ff6b40513c0 0x7ff6b4053880 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7ff6c800b000 tx=0x7ff6c80091a0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:27.594 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.580+0000 7ff6c7fff700 1 -- 172.21.15.40:0/2278826988 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7ff6b8042320 con 0x7ff6d0154900 2020-11-20T13:56:27.793 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.793+0000 7ff6d5188700 1 -- 172.21.15.40:0/2278826988 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6d00b9690 con 0x7ff6d0154900 2020-11-20T13:56:27.794 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.793+0000 7ff6c7fff700 1 -- 172.21.15.40:0/2278826988 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+2134 (secure 0 0 0) 0x7ff6b8045940 con 0x7ff6d0154900 2020-11-20T13:56:27.794 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:56:27.794 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/2019740031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2019740031},{"type":"v1","addr":"172.21.15.116:6835","nonce":2019740031}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T13:56:25.428178+0000","modified":"2020-11-20T13:56:27.432735+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6833/584131564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":584131564},{"type":"v1","addr":"172.21.15.40:6833","nonce":584131564}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-20T13:56:27.795 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.795+0000 7ff6d5188700 1 -- 172.21.15.40:0/2278826988 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7ff6b40513c0 msgr2=0x7ff6b4053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:27.795 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.795+0000 7ff6d5188700 1 --2- 172.21.15.40:0/2278826988 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7ff6b40513c0 0x7ff6b4053880 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7ff6c800b000 tx=0x7ff6c80091a0).stop 2020-11-20T13:56:27.796 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.795+0000 7ff6d5188700 1 -- 172.21.15.40:0/2278826988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6d0154900 msgr2=0x7ff6d0154d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:27.796 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.795+0000 7ff6d5188700 1 --2- 172.21.15.40:0/2278826988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6d0154900 0x7ff6d0154d80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7ff6b80013e0 tx=0x7ff6b8015000).stop 2020-11-20T13:56:27.796 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.795+0000 7ff6d5188700 1 -- 172.21.15.40:0/2278826988 shutdown_connections 2020-11-20T13:56:27.796 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.795+0000 7ff6d5188700 1 --2- 172.21.15.40:0/2278826988 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7ff6b40513c0 0x7ff6b4053880 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:27.797 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.795+0000 7ff6d5188700 1 --2- 172.21.15.40:0/2278826988 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff6d00b77b0 0x7ff6d00b7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:27.797 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.795+0000 7ff6d5188700 1 --2- 172.21.15.40:0/2278826988 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6d00b5790 0x7ff6d0142560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:27.797 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.795+0000 7ff6d5188700 1 --2- 172.21.15.40:0/2278826988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6d0154900 0x7ff6d0154d80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:27.797 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.795+0000 7ff6d5188700 1 -- 172.21.15.40:0/2278826988 >> 172.21.15.40:0/2278826988 conn(0x7ff6d00ba380 msgr2=0x7ff6d005f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:27.797 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.795+0000 7ff6d5188700 1 -- 172.21.15.40:0/2278826988 shutdown_connections 2020-11-20T13:56:27.798 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.795+0000 7ff6d5188700 1 -- 172.21.15.40:0/2278826988 wait complete. 2020-11-20T13:56:27.798 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 7 2020-11-20T13:56:27.806 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-20T13:56:27.920 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.920+0000 7f9eeb36b700 1 Processor -- start 2020-11-20T13:56:27.920 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.920+0000 7f9eeb36b700 1 -- start start 2020-11-20T13:56:27.923 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.921+0000 7f9eeb36b700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9ee409af20 0x7f9ee40b4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:27.923 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.921+0000 7f9eeb36b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9ee40b81c0 0x7f9ee40b4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:27.923 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.921+0000 7f9eeb36b700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ee409bff0 con 0x7f9ee409bc10 2020-11-20T13:56:27.924 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.921+0000 7f9eeb36b700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ee40b8740 con 0x7f9ee40b81c0 2020-11-20T13:56:27.924 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.921+0000 7f9eeb36b700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ee40b89e0 con 0x7f9ee409af20 2020-11-20T13:56:27.924 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.921+0000 7f9ee9908700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9ee40b81c0 0x7f9ee40b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:27.924 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.921+0000 7f9ee9908700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9ee40b81c0 0x7f9ee40b4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54552/0 (socket says 172.21.15.40:54552) 2020-11-20T13:56:27.925 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.921+0000 7f9ee9908700 1 -- 172.21.15.40:0/648800744 learned_addr learned my addr 172.21.15.40:0/648800744 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:27.925 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.921+0000 7f9ee8906700 1 --2- 172.21.15.40:0/648800744 >> v2:172.21.15.116:3300/0 conn(0x7f9ee409af20 0x7f9ee40b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:27.925 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.921+0000 7f9ee9908700 1 -- 172.21.15.40:0/648800744 >> v2:172.21.15.116:3300/0 conn(0x7f9ee409af20 msgr2=0x7f9ee40b4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:27.925 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.921+0000 7f9ee9908700 1 --2- 172.21.15.40:0/648800744 >> v2:172.21.15.116:3300/0 conn(0x7f9ee409af20 0x7f9ee40b4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:27.925 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.921+0000 7f9ee9908700 1 -- 172.21.15.40:0/648800744 >> v1:172.21.15.116:6789/0 conn(0x7f9ee409bc10 legacy=0x7f9ee40aff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:27.926 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.921+0000 7f9ee9908700 1 -- 172.21.15.40:0/648800744 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee40b8c80 con 0x7f9ee40b81c0 2020-11-20T13:56:27.926 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.921+0000 7f9ee9908700 1 --2- 172.21.15.40:0/648800744 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ee40b81c0 0x7f9ee40b4740 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f9ee40afe50 tx=0x7f9edc009270).ready entity=mon.0 client_cookie=77151b8e3935a516 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:27.926 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.922+0000 7f9ed3fff700 1 -- 172.21.15.40:0/648800744 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9edc00bbc0 con 0x7f9ee40b81c0 2020-11-20T13:56:27.926 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.922+0000 7f9ed3fff700 1 -- 172.21.15.40:0/648800744 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9edc00bd20 con 0x7f9ee40b81c0 2020-11-20T13:56:27.926 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.922+0000 7f9ed3fff700 1 -- 172.21.15.40:0/648800744 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9edc011400 con 0x7f9ee40b81c0 2020-11-20T13:56:27.927 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.922+0000 7f9eeb36b700 1 -- 172.21.15.40:0/648800744 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ee40b81c0 msgr2=0x7f9ee40b4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:27.927 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.922+0000 7f9eeb36b700 1 --2- 172.21.15.40:0/648800744 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ee40b81c0 0x7f9ee40b4740 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f9ee40afe50 tx=0x7f9edc009270).stop 2020-11-20T13:56:27.927 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.922+0000 7f9eeb36b700 1 -- 172.21.15.40:0/648800744 shutdown_connections 2020-11-20T13:56:27.927 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.922+0000 7f9eeb36b700 1 --2- 172.21.15.40:0/648800744 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ee40b81c0 0x7f9ee40b4740 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:27.927 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.922+0000 7f9eeb36b700 1 --2- 172.21.15.40:0/648800744 >> v2:172.21.15.116:3300/0 conn(0x7f9ee409af20 0x7f9ee40b4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:27.928 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.922+0000 7f9eeb36b700 1 -- 172.21.15.40:0/648800744 >> 172.21.15.40:0/648800744 conn(0x7f9ee40ba380 msgr2=0x7f9ee40b6350 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:27.928 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.922+0000 7f9eeb36b700 1 -- 172.21.15.40:0/648800744 shutdown_connections 2020-11-20T13:56:27.928 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.922+0000 7f9eeb36b700 1 -- 172.21.15.40:0/648800744 wait complete. 2020-11-20T13:56:27.931 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.923+0000 7f9eeb36b700 1 Processor -- start 2020-11-20T13:56:27.931 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.923+0000 7f9eeb36b700 1 -- start start 2020-11-20T13:56:27.932 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.924+0000 7f9eeb36b700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9ee409af20 0x7f9ee409e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:27.932 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.924+0000 7f9eeb36b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ee409bc10 0x7f9ee4156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:27.932 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.924+0000 7f9eeb36b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ee40b81c0 0x7f9ee413ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:27.932 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.924+0000 7f9eeb36b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee4156d40 con 0x7f9ee40b81c0 2020-11-20T13:56:27.932 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.924+0000 7f9eeb36b700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee4156fe0 con 0x7f9ee409bc10 2020-11-20T13:56:27.933 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.924+0000 7f9eeb36b700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee4157280 con 0x7f9ee409af20 2020-11-20T13:56:27.933 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.924+0000 7f9ee9908700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ee40b81c0 0x7f9ee413ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:27.934 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.924+0000 7f9ee9908700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ee40b81c0 0x7f9ee413ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54556/0 (socket says 172.21.15.40:54556) 2020-11-20T13:56:27.934 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.924+0000 7f9ee9908700 1 -- 172.21.15.40:0/1501706597 learned_addr learned my addr 172.21.15.40:0/1501706597 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:27.934 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.924+0000 7f9ee8906700 1 --2- 172.21.15.40:0/1501706597 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ee409bc10 0x7f9ee4156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:27.934 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.924+0000 7f9ee9107700 1 --2- 172.21.15.40:0/1501706597 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9ee409af20 0x7f9ee409e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:27.935 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.924+0000 7f9ee9908700 1 -- 172.21.15.40:0/1501706597 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9ee409af20 msgr2=0x7f9ee409e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:27.935 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.924+0000 7f9ee9908700 1 --2- 172.21.15.40:0/1501706597 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9ee409af20 0x7f9ee409e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:27.935 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.924+0000 7f9ee9908700 1 -- 172.21.15.40:0/1501706597 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ee409bc10 msgr2=0x7f9ee4156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:27.936 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.924+0000 7f9ee9908700 1 --2- 172.21.15.40:0/1501706597 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ee409bc10 0x7f9ee4156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:27.936 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.924+0000 7f9ee9908700 1 -- 172.21.15.40:0/1501706597 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee4156a40 con 0x7f9ee40b81c0 2020-11-20T13:56:27.936 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.924+0000 7f9ee9908700 1 --2- 172.21.15.40:0/1501706597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ee40b81c0 0x7f9ee413ca00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f9ee40afe50 tx=0x7f9edc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:27.936 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.924+0000 7f9ed27fc700 1 -- 172.21.15.40:0/1501706597 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9edc01e420 con 0x7f9ee40b81c0 2020-11-20T13:56:27.936 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.924+0000 7f9ed27fc700 1 -- 172.21.15.40:0/1501706597 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9edc01e580 con 0x7f9ee40b81c0 2020-11-20T13:56:27.937 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.924+0000 7f9eeb36b700 1 -- 172.21.15.40:0/1501706597 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ee4143e50 con 0x7f9ee40b81c0 2020-11-20T13:56:27.937 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.925+0000 7f9ed27fc700 1 -- 172.21.15.40:0/1501706597 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9edc01e8e0 con 0x7f9ee40b81c0 2020-11-20T13:56:27.937 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.925+0000 7f9eeb36b700 1 -- 172.21.15.40:0/1501706597 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ee413cef0 con 0x7f9ee40b81c0 2020-11-20T13:56:27.937 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.926+0000 7f9ed27fc700 1 -- 172.21.15.40:0/1501706597 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f9edc00f070 con 0x7f9ee40b81c0 2020-11-20T13:56:27.937 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.926+0000 7f9eeb36b700 1 -- 172.21.15.40:0/1501706597 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ee4157440 con 0x7f9ee40b81c0 2020-11-20T13:56:27.938 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.926+0000 7f9ed27fc700 1 --2- 172.21.15.40:0/1501706597 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f9ec4051290 0x7f9ec4053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:27.938 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.926+0000 7f9ed27fc700 1 -- 172.21.15.40:0/1501706597 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f9edc06a830 con 0x7f9ee40b81c0 2020-11-20T13:56:27.938 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.926+0000 7f9ee9107700 1 --2- 172.21.15.40:0/1501706597 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f9ec4051290 0x7f9ec4053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:27.938 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.926+0000 7f9ee9107700 1 --2- 172.21.15.40:0/1501706597 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f9ec4051290 0x7f9ec4053750 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f9ed400d000 tx=0x7f9ed4007ab0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:27.938 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:27.929+0000 7f9ed27fc700 1 -- 172.21.15.40:0/1501706597 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f9edc042a50 con 0x7f9ee40b81c0 2020-11-20T13:56:28.091 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.091+0000 7f9eeb36b700 1 -- 172.21.15.40:0/1501706597 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ee4157440 con 0x7f9ee40b81c0 2020-11-20T13:56:28.092 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.092+0000 7f9ed27fc700 1 -- 172.21.15.40:0/1501706597 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16347 (secure 0 0 0) 0x7f9edc015090 con 0x7f9ee40b81c0 2020-11-20T13:56:28.092 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:56:28.092 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":24,"fsid":"da1ed737-5174-430f-a3c9-56f6bbfad13c","created":"2020-11-20T13:56:04.990651+0000","modified":"2020-11-20T13:56:25.428032+0000","last_up_change":"2020-11-20T13:56:13.893026+0000","last_in_change":"2020-11-20T13:56:13.893026+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T13:56:08.330017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T13:56:17.939684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T13:56:20.563740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T13:56:23.062623+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9ea4c667-8938-46d6-a5ca-092d55c1f79e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":22825},{"type":"v1","addr":"172.21.15.40:6818","nonce":22825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":22825},{"type":"v1","addr":"172.21.15.40:6822","nonce":22825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":22825},{"type":"v1","addr":"172.21.15.40:6830","nonce":22825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":22825},{"type":"v1","addr":"172.21.15.40:6826","nonce":22825}]},"public_addr":"172.21.15.40:6818/22825","cluster_addr":"172.21.15.40:6822/22825","heartbeat_back_addr":"172.21.15.40:6830/22825","heartbeat_front_addr":"172.21.15.40:6826/22825","state":["exists","up"]},{"osd":1,"uuid":"515708d9-5087-4e55-8d14-cdea1c00eacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":22823},{"type":"v1","addr":"172.21.15.40:6801","nonce":22823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":22823},{"type":"v1","addr":"172.21.15.40:6803","nonce":22823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":22823},{"type":"v1","addr":"172.21.15.40:6807","nonce":22823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":22823},{"type":"v1","addr":"172.21.15.40:6805","nonce":22823}]},"public_addr":"172.21.15.40:6801/22823","cluster_addr":"172.21.15.40:6803/22823","heartbeat_back_addr":"172.21.15.40:6807/22823","heartbeat_front_addr":"172.21.15.40:6805/22823","state":["exists","up"]},{"osd":2,"uuid":"4d86bca7-743e-4983-b35f-36676c405b36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":22824},{"type":"v1","addr":"172.21.15.40:6809","nonce":22824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":22824},{"type":"v1","addr":"172.21.15.40:6811","nonce":22824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":22824},{"type":"v1","addr":"172.21.15.40:6815","nonce":22824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":22824},{"type":"v1","addr":"172.21.15.40:6813","nonce":22824}]},"public_addr":"172.21.15.40:6809/22824","cluster_addr":"172.21.15.40:6811/22824","heartbeat_back_addr":"172.21.15.40:6815/22824","heartbeat_front_addr":"172.21.15.40:6813/22824","state":["exists","up"]},{"osd":3,"uuid":"f4dc726b-6b34-4e8b-8eeb-87060b58002d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":22826},{"type":"v1","addr":"172.21.15.40:6819","nonce":22826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":22826},{"type":"v1","addr":"172.21.15.40:6823","nonce":22826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":22826},{"type":"v1","addr":"172.21.15.40:6831","nonce":22826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":22826},{"type":"v1","addr":"172.21.15.40:6827","nonce":22826}]},"public_addr":"172.21.15.40:6819/22826","cluster_addr":"172.21.15.40:6823/22826","heartbeat_back_addr":"172.21.15.40:6831/22826","heartbeat_front_addr":"172.21.15.40:6827/22826","state":["exists","up"]},{"osd":4,"uuid":"25a6d53b-d1f0-4498-b7d5-ffa38661d403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":23181},{"type":"v1","addr":"172.21.15.116:6819","nonce":23181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":23181},{"type":"v1","addr":"172.21.15.116:6821","nonce":23181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":23181},{"type":"v1","addr":"172.21.15.116:6825","nonce":23181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":23181},{"type":"v1","addr":"172.21.15.116:6823","nonce":23181}]},"public_addr":"172.21.15.116:6819/23181","cluster_addr":"172.21.15.116:6821/23181","heartbeat_back_addr":"172.21.15.116:6825/23181","heartbeat_front_addr":"172.21.15.116:6823/23181","state":["exists","up"]},{"osd":5,"uuid":"7d7a5af8-229b-4351-9a21-a3d509cd4635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":23171},{"type":"v1","addr":"172.21.15.116:6803","nonce":23171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":23171},{"type":"v1","addr":"172.21.15.116:6805","nonce":23171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":23171},{"type":"v1","addr":"172.21.15.116:6809","nonce":23171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":23171},{"type":"v1","addr":"172.21.15.116:6807","nonce":23171}]},"public_addr":"172.21.15.116:6803/23171","cluster_addr":"172.21.15.116:6805/23171","heartbeat_back_addr":"172.21.15.116:6809/23171","heartbeat_front_addr":"172.21.15.116:6807/23171","state":["exists","up"]},{"osd":6,"uuid":"8247d248-a140-4e36-9796-9847a97df58d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":23172},{"type":"v1","addr":"172.21.15.116:6811","nonce":23172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":23172},{"type":"v1","addr":"172.21.15.116:6813","nonce":23172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":23172},{"type":"v1","addr":"172.21.15.116:6817","nonce":23172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":23172},{"type":"v1","addr":"172.21.15.116:6815","nonce":23172}]},"public_addr":"172.21.15.116:6811/23172","cluster_addr":"172.21.15.116:6813/23172","heartbeat_back_addr":"172.21.15.116:6817/23172","heartbeat_front_addr":"172.21.15.116:6815/23172","state":["exists","up"]},{"osd":7,"uuid":"472803c4-70b8-499d-b18a-71aaeec6aab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":23196},{"type":"v1","addr":"172.21.15.116:6827","nonce":23196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":23196},{"type":"v1","addr":"172.21.15.116:6829","nonce":23196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":23196},{"type":"v1","addr":"172.21.15.116:6833","nonce":23196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":23196},{"type":"v1","addr":"172.21.15.116:6831","nonce":23196}]},"public_addr":"172.21.15.116:6827/23196","cluster_addr":"172.21.15.116:6829/23196","heartbeat_back_addr":"172.21.15.116:6833/23196","heartbeat_front_addr":"172.21.15.116:6831/23196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.029357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:11.989797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.091097+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.014766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.070860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.071864+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.063400+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.109028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-20T13:56:28.093 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.093+0000 7f9eeb36b700 1 -- 172.21.15.40:0/1501706597 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f9ec4051290 msgr2=0x7f9ec4053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:28.094 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.093+0000 7f9eeb36b700 1 --2- 172.21.15.40:0/1501706597 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f9ec4051290 0x7f9ec4053750 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f9ed400d000 tx=0x7f9ed4007ab0).stop 2020-11-20T13:56:28.094 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.093+0000 7f9eeb36b700 1 -- 172.21.15.40:0/1501706597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ee40b81c0 msgr2=0x7f9ee413ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:28.094 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.093+0000 7f9eeb36b700 1 --2- 172.21.15.40:0/1501706597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ee40b81c0 0x7f9ee413ca00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f9ee40afe50 tx=0x7f9edc016000).stop 2020-11-20T13:56:28.094 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.093+0000 7f9eeb36b700 1 -- 172.21.15.40:0/1501706597 shutdown_connections 2020-11-20T13:56:28.095 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.093+0000 7f9eeb36b700 1 --2- 172.21.15.40:0/1501706597 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f9ec4051290 0x7f9ec4053750 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.095 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.093+0000 7f9eeb36b700 1 --2- 172.21.15.40:0/1501706597 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9ee409af20 0x7f9ee409e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.095 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.093+0000 7f9eeb36b700 1 --2- 172.21.15.40:0/1501706597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ee40b81c0 0x7f9ee413ca00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.096 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.093+0000 7f9eeb36b700 1 --2- 172.21.15.40:0/1501706597 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ee409bc10 0x7f9ee4156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.096 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.093+0000 7f9eeb36b700 1 -- 172.21.15.40:0/1501706597 >> 172.21.15.40:0/1501706597 conn(0x7f9ee40ba380 msgr2=0x7f9ee405fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:28.096 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.093+0000 7f9eeb36b700 1 -- 172.21.15.40:0/1501706597 shutdown_connections 2020-11-20T13:56:28.096 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.093+0000 7f9eeb36b700 1 -- 172.21.15.40:0/1501706597 wait complete. 2020-11-20T13:56:28.104 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-20T13:56:28.105 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2020-11-20T13:56:28.219 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.219+0000 7fab77d45700 1 Processor -- start 2020-11-20T13:56:28.219 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.219+0000 7fab77d45700 1 -- start start 2020-11-20T13:56:28.221 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.219+0000 7fab77d45700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fab7009bc10 0x7fab700aff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:28.221 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.219+0000 7fab77d45700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fab700b81c0 0x7fab700b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:28.222 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.219+0000 7fab77d45700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fab70099f10 0x7fab7009a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:28.222 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.219+0000 7fab77d45700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab7009bff0 con 0x7fab7009bc10 2020-11-20T13:56:28.222 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.219+0000 7fab77d45700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab700b8740 con 0x7fab70099f10 2020-11-20T13:56:28.222 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.219+0000 7fab77d45700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab700b89e0 con 0x7fab700b81c0 2020-11-20T13:56:28.223 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.220+0000 7fab75ae1700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fab7009bc10 0x7fab700aff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:28.223 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.220+0000 7fab75ae1700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fab7009bc10 0x7fab700aff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54562/0 (socket says 172.21.15.40:54562) 2020-11-20T13:56:28.223 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.220+0000 7fab75ae1700 1 -- 172.21.15.40:0/8271308 learned_addr learned my addr 172.21.15.40:0/8271308 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:28.223 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.220+0000 7fab752e0700 1 --2- 172.21.15.40:0/8271308 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fab700b81c0 0x7fab700b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:28.223 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.220+0000 7fab762e2700 1 --2- 172.21.15.40:0/8271308 >> v2:172.21.15.116:3300/0 conn(0x7fab70099f10 0x7fab7009a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.220+0000 7fab75ae1700 1 -- 172.21.15.40:0/8271308 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fab700b81c0 msgr2=0x7fab700b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.220+0000 7fab75ae1700 1 --2- 172.21.15.40:0/8271308 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fab700b81c0 0x7fab700b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.220+0000 7fab75ae1700 1 -- 172.21.15.40:0/8271308 >> v2:172.21.15.116:3300/0 conn(0x7fab70099f10 msgr2=0x7fab7009a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.220+0000 7fab75ae1700 1 --2- 172.21.15.40:0/8271308 >> v2:172.21.15.116:3300/0 conn(0x7fab70099f10 0x7fab7009a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.220+0000 7fab75ae1700 1 -- 172.21.15.40:0/8271308 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab700b8c80 con 0x7fab7009bc10 2020-11-20T13:56:28.225 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.220+0000 7fab75ae1700 1 --2- 172.21.15.40:0/8271308 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab7009bc10 0x7fab700aff40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fab60002f30 tx=0x7fab60006e80).ready entity=mon.0 client_cookie=ea78f8c4ae4838f2 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:28.225 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.220+0000 7fab74adf700 1 -- 172.21.15.40:0/8271308 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab600097f0 con 0x7fab7009bc10 2020-11-20T13:56:28.225 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.220+0000 7fab74adf700 1 -- 172.21.15.40:0/8271308 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab60009950 con 0x7fab7009bc10 2020-11-20T13:56:28.225 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.220+0000 7fab74adf700 1 -- 172.21.15.40:0/8271308 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab60009cb0 con 0x7fab7009bc10 2020-11-20T13:56:28.225 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.220+0000 7fab77d45700 1 -- 172.21.15.40:0/8271308 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab7009bc10 msgr2=0x7fab700aff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:28.226 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.220+0000 7fab77d45700 1 --2- 172.21.15.40:0/8271308 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab7009bc10 0x7fab700aff40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fab60002f30 tx=0x7fab60006e80).stop 2020-11-20T13:56:28.226 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.221+0000 7fab77d45700 1 -- 172.21.15.40:0/8271308 shutdown_connections 2020-11-20T13:56:28.226 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.221+0000 7fab77d45700 1 --2- 172.21.15.40:0/8271308 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab7009bc10 0x7fab700aff40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.226 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.221+0000 7fab77d45700 1 --2- 172.21.15.40:0/8271308 >> v2:172.21.15.116:3300/0 conn(0x7fab70099f10 0x7fab7009a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.226 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.221+0000 7fab77d45700 1 --2- 172.21.15.40:0/8271308 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fab700b81c0 0x7fab700b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.227 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.221+0000 7fab77d45700 1 -- 172.21.15.40:0/8271308 >> 172.21.15.40:0/8271308 conn(0x7fab700ba380 msgr2=0x7fab700b6350 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:28.227 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.221+0000 7fab77d45700 1 -- 172.21.15.40:0/8271308 shutdown_connections 2020-11-20T13:56:28.227 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.221+0000 7fab77d45700 1 -- 172.21.15.40:0/8271308 wait complete. 2020-11-20T13:56:28.230 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.222+0000 7fab77d45700 1 Processor -- start 2020-11-20T13:56:28.230 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.222+0000 7fab77d45700 1 -- start start 2020-11-20T13:56:28.230 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.222+0000 7fab77d45700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab70099f10 0x7fab7009ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:28.231 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.222+0000 7fab77d45700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab7009bc10 0x7fab7013cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:28.231 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.222+0000 7fab77d45700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fab700b81c0 0x7fab7013d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:28.231 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.222+0000 7fab77d45700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7009e4f0 con 0x7fab7009bc10 2020-11-20T13:56:28.231 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.222+0000 7fab77d45700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7009e750 con 0x7fab70099f10 2020-11-20T13:56:28.232 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.222+0000 7fab77d45700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab700b4dc0 con 0x7fab700b81c0 2020-11-20T13:56:28.232 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.223+0000 7fab752e0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab7009bc10 0x7fab7013cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:28.232 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.223+0000 7fab752e0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab7009bc10 0x7fab7013cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54570/0 (socket says 172.21.15.40:54570) 2020-11-20T13:56:28.232 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.223+0000 7fab752e0700 1 -- 172.21.15.40:0/3416021648 learned_addr learned my addr 172.21.15.40:0/3416021648 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:28.232 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.223+0000 7fab75ae1700 1 --2- 172.21.15.40:0/3416021648 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab70099f10 0x7fab7009ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:28.233 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.223+0000 7fab762e2700 1 --2- 172.21.15.40:0/3416021648 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fab700b81c0 0x7fab7013d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:28.233 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.223+0000 7fab752e0700 1 -- 172.21.15.40:0/3416021648 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fab700b81c0 msgr2=0x7fab7013d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:28.233 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.223+0000 7fab752e0700 1 --2- 172.21.15.40:0/3416021648 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fab700b81c0 0x7fab7013d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.233 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.223+0000 7fab752e0700 1 -- 172.21.15.40:0/3416021648 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab70099f10 msgr2=0x7fab7009ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:28.233 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.223+0000 7fab752e0700 1 --2- 172.21.15.40:0/3416021648 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab70099f10 0x7fab7009ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.234 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.223+0000 7fab752e0700 1 -- 172.21.15.40:0/3416021648 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab7012e410 con 0x7fab7009bc10 2020-11-20T13:56:28.234 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.223+0000 7fab752e0700 1 --2- 172.21.15.40:0/3416021648 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab7009bc10 0x7fab7013cb40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fab6c003070 tx=0x7fab6c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:28.234 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.223+0000 7fab5effd700 1 -- 172.21.15.40:0/3416021648 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab6c009790 con 0x7fab7009bc10 2020-11-20T13:56:28.234 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.223+0000 7fab5effd700 1 -- 172.21.15.40:0/3416021648 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab6c0098f0 con 0x7fab7009bc10 2020-11-20T13:56:28.234 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.223+0000 7fab77d45700 1 -- 172.21.15.40:0/3416021648 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab7009f0e0 con 0x7fab7009bc10 2020-11-20T13:56:28.235 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.223+0000 7fab5effd700 1 -- 172.21.15.40:0/3416021648 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab6c009c50 con 0x7fab7009bc10 2020-11-20T13:56:28.235 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.223+0000 7fab77d45700 1 -- 172.21.15.40:0/3416021648 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab701462d0 con 0x7fab7009bc10 2020-11-20T13:56:28.235 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.224+0000 7fab5effd700 1 -- 172.21.15.40:0/3416021648 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fab6c013480 con 0x7fab7009bc10 2020-11-20T13:56:28.235 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.224+0000 7fab77d45700 1 -- 172.21.15.40:0/3416021648 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab70057050 con 0x7fab7009bc10 2020-11-20T13:56:28.235 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.225+0000 7fab5effd700 1 --2- 172.21.15.40:0/3416021648 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fab50051210 0x7fab500536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:28.236 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.225+0000 7fab5effd700 1 -- 172.21.15.40:0/3416021648 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fab6c0694b0 con 0x7fab7009bc10 2020-11-20T13:56:28.236 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.225+0000 7fab75ae1700 1 --2- 172.21.15.40:0/3416021648 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fab50051210 0x7fab500536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:28.236 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.225+0000 7fab75ae1700 1 --2- 172.21.15.40:0/3416021648 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fab50051210 0x7fab500536d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fab6001c000 tx=0x7fab60015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:28.236 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.228+0000 7fab5effd700 1 -- 172.21.15.40:0/3416021648 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fab6c041670 con 0x7fab7009bc10 2020-11-20T13:56:28.388 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.388+0000 7fab77d45700 1 -- 172.21.15.40:0/3416021648 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab7009e8c0 con 0x7fab7009bc10 2020-11-20T13:56:28.389 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.389+0000 7fab5effd700 1 -- 172.21.15.40:0/3416021648 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16347 (secure 0 0 0) 0x7fab6c009db0 con 0x7fab7009bc10 2020-11-20T13:56:28.389 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:56:28.389 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":24,"fsid":"da1ed737-5174-430f-a3c9-56f6bbfad13c","created":"2020-11-20T13:56:04.990651+0000","modified":"2020-11-20T13:56:25.428032+0000","last_up_change":"2020-11-20T13:56:13.893026+0000","last_in_change":"2020-11-20T13:56:13.893026+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T13:56:08.330017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T13:56:17.939684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T13:56:20.563740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T13:56:23.062623+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9ea4c667-8938-46d6-a5ca-092d55c1f79e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":22825},{"type":"v1","addr":"172.21.15.40:6818","nonce":22825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":22825},{"type":"v1","addr":"172.21.15.40:6822","nonce":22825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":22825},{"type":"v1","addr":"172.21.15.40:6830","nonce":22825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":22825},{"type":"v1","addr":"172.21.15.40:6826","nonce":22825}]},"public_addr":"172.21.15.40:6818/22825","cluster_addr":"172.21.15.40:6822/22825","heartbeat_back_addr":"172.21.15.40:6830/22825","heartbeat_front_addr":"172.21.15.40:6826/22825","state":["exists","up"]},{"osd":1,"uuid":"515708d9-5087-4e55-8d14-cdea1c00eacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":22823},{"type":"v1","addr":"172.21.15.40:6801","nonce":22823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":22823},{"type":"v1","addr":"172.21.15.40:6803","nonce":22823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":22823},{"type":"v1","addr":"172.21.15.40:6807","nonce":22823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":22823},{"type":"v1","addr":"172.21.15.40:6805","nonce":22823}]},"public_addr":"172.21.15.40:6801/22823","cluster_addr":"172.21.15.40:6803/22823","heartbeat_back_addr":"172.21.15.40:6807/22823","heartbeat_front_addr":"172.21.15.40:6805/22823","state":["exists","up"]},{"osd":2,"uuid":"4d86bca7-743e-4983-b35f-36676c405b36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":22824},{"type":"v1","addr":"172.21.15.40:6809","nonce":22824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":22824},{"type":"v1","addr":"172.21.15.40:6811","nonce":22824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":22824},{"type":"v1","addr":"172.21.15.40:6815","nonce":22824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":22824},{"type":"v1","addr":"172.21.15.40:6813","nonce":22824}]},"public_addr":"172.21.15.40:6809/22824","cluster_addr":"172.21.15.40:6811/22824","heartbeat_back_addr":"172.21.15.40:6815/22824","heartbeat_front_addr":"172.21.15.40:6813/22824","state":["exists","up"]},{"osd":3,"uuid":"f4dc726b-6b34-4e8b-8eeb-87060b58002d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":22826},{"type":"v1","addr":"172.21.15.40:6819","nonce":22826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":22826},{"type":"v1","addr":"172.21.15.40:6823","nonce":22826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":22826},{"type":"v1","addr":"172.21.15.40:6831","nonce":22826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":22826},{"type":"v1","addr":"172.21.15.40:6827","nonce":22826}]},"public_addr":"172.21.15.40:6819/22826","cluster_addr":"172.21.15.40:6823/22826","heartbeat_back_addr":"172.21.15.40:6831/22826","heartbeat_front_addr":"172.21.15.40:6827/22826","state":["exists","up"]},{"osd":4,"uuid":"25a6d53b-d1f0-4498-b7d5-ffa38661d403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":23181},{"type":"v1","addr":"172.21.15.116:6819","nonce":23181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":23181},{"type":"v1","addr":"172.21.15.116:6821","nonce":23181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":23181},{"type":"v1","addr":"172.21.15.116:6825","nonce":23181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":23181},{"type":"v1","addr":"172.21.15.116:6823","nonce":23181}]},"public_addr":"172.21.15.116:6819/23181","cluster_addr":"172.21.15.116:6821/23181","heartbeat_back_addr":"172.21.15.116:6825/23181","heartbeat_front_addr":"172.21.15.116:6823/23181","state":["exists","up"]},{"osd":5,"uuid":"7d7a5af8-229b-4351-9a21-a3d509cd4635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":23171},{"type":"v1","addr":"172.21.15.116:6803","nonce":23171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":23171},{"type":"v1","addr":"172.21.15.116:6805","nonce":23171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":23171},{"type":"v1","addr":"172.21.15.116:6809","nonce":23171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":23171},{"type":"v1","addr":"172.21.15.116:6807","nonce":23171}]},"public_addr":"172.21.15.116:6803/23171","cluster_addr":"172.21.15.116:6805/23171","heartbeat_back_addr":"172.21.15.116:6809/23171","heartbeat_front_addr":"172.21.15.116:6807/23171","state":["exists","up"]},{"osd":6,"uuid":"8247d248-a140-4e36-9796-9847a97df58d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":23172},{"type":"v1","addr":"172.21.15.116:6811","nonce":23172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":23172},{"type":"v1","addr":"172.21.15.116:6813","nonce":23172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":23172},{"type":"v1","addr":"172.21.15.116:6817","nonce":23172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":23172},{"type":"v1","addr":"172.21.15.116:6815","nonce":23172}]},"public_addr":"172.21.15.116:6811/23172","cluster_addr":"172.21.15.116:6813/23172","heartbeat_back_addr":"172.21.15.116:6817/23172","heartbeat_front_addr":"172.21.15.116:6815/23172","state":["exists","up"]},{"osd":7,"uuid":"472803c4-70b8-499d-b18a-71aaeec6aab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":23196},{"type":"v1","addr":"172.21.15.116:6827","nonce":23196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":23196},{"type":"v1","addr":"172.21.15.116:6829","nonce":23196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":23196},{"type":"v1","addr":"172.21.15.116:6833","nonce":23196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":23196},{"type":"v1","addr":"172.21.15.116:6831","nonce":23196}]},"public_addr":"172.21.15.116:6827/23196","cluster_addr":"172.21.15.116:6829/23196","heartbeat_back_addr":"172.21.15.116:6833/23196","heartbeat_front_addr":"172.21.15.116:6831/23196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.029357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:11.989797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.091097+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.014766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.070860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.071864+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.063400+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.109028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-20T13:56:28.391 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.390+0000 7fab77d45700 1 -- 172.21.15.40:0/3416021648 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fab50051210 msgr2=0x7fab500536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:28.391 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.390+0000 7fab77d45700 1 --2- 172.21.15.40:0/3416021648 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fab50051210 0x7fab500536d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fab6001c000 tx=0x7fab60015000).stop 2020-11-20T13:56:28.391 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.390+0000 7fab77d45700 1 -- 172.21.15.40:0/3416021648 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab7009bc10 msgr2=0x7fab7013cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:28.391 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.390+0000 7fab77d45700 1 --2- 172.21.15.40:0/3416021648 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab7009bc10 0x7fab7013cb40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fab6c003070 tx=0x7fab6c008040).stop 2020-11-20T13:56:28.391 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.390+0000 7fab77d45700 1 -- 172.21.15.40:0/3416021648 shutdown_connections 2020-11-20T13:56:28.392 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.390+0000 7fab77d45700 1 --2- 172.21.15.40:0/3416021648 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fab50051210 0x7fab500536d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.392 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.390+0000 7fab77d45700 1 --2- 172.21.15.40:0/3416021648 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fab700b81c0 0x7fab7013d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.392 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.390+0000 7fab77d45700 1 --2- 172.21.15.40:0/3416021648 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab70099f10 0x7fab7009ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.392 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.390+0000 7fab77d45700 1 --2- 172.21.15.40:0/3416021648 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab7009bc10 0x7fab7013cb40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.392 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.390+0000 7fab77d45700 1 -- 172.21.15.40:0/3416021648 >> 172.21.15.40:0/3416021648 conn(0x7fab700ba380 msgr2=0x7fab7013d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:28.393 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.390+0000 7fab77d45700 1 -- 172.21.15.40:0/3416021648 shutdown_connections 2020-11-20T13:56:28.393 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.391+0000 7fab77d45700 1 -- 172.21.15.40:0/3416021648 wait complete. 2020-11-20T13:56:28.401 INFO:tasks.ceph.ceph_manager.ceph:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T13:56:08.330017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T13:56:17.939684+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T13:56:20.563740+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T13:56:23.062623+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T13:56:28.402 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-20T13:56:28.520 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.520+0000 7f1692255700 1 Processor -- start 2020-11-20T13:56:28.521 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.520+0000 7f1692255700 1 -- start start 2020-11-20T13:56:28.523 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.521+0000 7f1692255700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f168c0b31f0 0x7f168c144bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:28.523 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.521+0000 7f1692255700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f168c0b1450 0x7f168c0b1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:28.523 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.521+0000 7f1692255700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f168c0b1df0 con 0x7f168c0b6c80 2020-11-20T13:56:28.524 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.521+0000 7f1692255700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f168c0b0d10 con 0x7f168c0b1450 2020-11-20T13:56:28.524 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.521+0000 7f1692255700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f168c0b0fb0 con 0x7f168c0b31f0 2020-11-20T13:56:28.524 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.521+0000 7f168b7fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f168c0b6c80 0x7f168c0b40f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45172/0 (socket says 172.21.15.40:45172) 2020-11-20T13:56:28.524 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.521+0000 7f168b7fe700 1 -- 172.21.15.40:0/610425443 learned_addr learned my addr 172.21.15.40:0/610425443 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:28.525 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.521+0000 7f168affd700 1 --2- 172.21.15.40:0/610425443 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f168c0b31f0 0x7f168c144bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:28.525 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.521+0000 7f168bfff700 1 --2- 172.21.15.40:0/610425443 >> v2:172.21.15.116:3300/0 conn(0x7f168c0b1450 0x7f168c0b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:28.525 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.521+0000 7f168a7fc700 1 -- 172.21.15.40:0/610425443 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4217397946 0 0) 0x7f168c0b1df0 con 0x7f168c0b6c80 2020-11-20T13:56:28.525 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.521+0000 7f168a7fc700 1 -- 172.21.15.40:0/610425443 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1678003230 con 0x7f168c0b6c80 2020-11-20T13:56:28.526 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.522+0000 7f168bfff700 1 -- 172.21.15.40:0/610425443 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f168c0b31f0 msgr2=0x7f168c144bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:28.526 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.522+0000 7f168bfff700 1 --2- 172.21.15.40:0/610425443 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f168c0b31f0 0x7f168c144bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.526 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.522+0000 7f168bfff700 1 -- 172.21.15.40:0/610425443 >> v1:172.21.15.40:6789/0 conn(0x7f168c0b6c80 legacy=0x7f168c0b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:28.526 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.522+0000 7f168a7fc700 1 -- 172.21.15.40:0/610425443 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1090711099 0 0) 0x7f1678003230 con 0x7f168c0b6c80 2020-11-20T13:56:28.526 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.522+0000 7f168bfff700 1 -- 172.21.15.40:0/610425443 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f168c0b7060 con 0x7f168c0b1450 2020-11-20T13:56:28.527 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.522+0000 7f168affd700 1 --2- 172.21.15.40:0/610425443 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f168c0b31f0 0x7f168c144bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:56:28.527 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.522+0000 7f168bfff700 1 --2- 172.21.15.40:0/610425443 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f168c0b1450 0x7f168c0b1900 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f167c0019f0 tx=0x7f16840091a0).ready entity=mon.1 client_cookie=61e9e0247b4f24bb server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:28.527 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.522+0000 7f168a7fc700 1 -- 172.21.15.40:0/610425443 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f167c001850 con 0x7f168c0b1450 2020-11-20T13:56:28.527 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.522+0000 7f168a7fc700 1 -- 172.21.15.40:0/610425443 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f168400b800 con 0x7f168c0b1450 2020-11-20T13:56:28.527 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.522+0000 7f168a7fc700 1 -- 172.21.15.40:0/610425443 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f168400b970 con 0x7f168c0b1450 2020-11-20T13:56:28.528 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.522+0000 7f1692255700 1 -- 172.21.15.40:0/610425443 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f168c0b1450 msgr2=0x7f168c0b1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:28.528 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.522+0000 7f1692255700 1 --2- 172.21.15.40:0/610425443 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f168c0b1450 0x7f168c0b1900 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f167c0019f0 tx=0x7f16840091a0).stop 2020-11-20T13:56:28.528 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.523+0000 7f1692255700 1 -- 172.21.15.40:0/610425443 shutdown_connections 2020-11-20T13:56:28.528 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.523+0000 7f1692255700 1 --2- 172.21.15.40:0/610425443 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f168c0b1450 0x7f168c0b1900 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.528 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.523+0000 7f1692255700 1 --2- 172.21.15.40:0/610425443 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f168c0b31f0 0x7f168c144bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.529 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.523+0000 7f1692255700 1 -- 172.21.15.40:0/610425443 >> 172.21.15.40:0/610425443 conn(0x7f168c0ba1d0 msgr2=0x7f168c0b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:28.529 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.523+0000 7f1692255700 1 -- 172.21.15.40:0/610425443 shutdown_connections 2020-11-20T13:56:28.529 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.523+0000 7f1692255700 1 -- 172.21.15.40:0/610425443 wait complete. 2020-11-20T13:56:28.533 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.524+0000 7f1692255700 1 Processor -- start 2020-11-20T13:56:28.533 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.524+0000 7f1692255700 1 -- start start 2020-11-20T13:56:28.533 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.525+0000 7f1692255700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f168c0b1450 0x7f168c143650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:28.533 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.525+0000 7f1692255700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f168c0b31f0 0x7f168c124130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:28.534 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.525+0000 7f1692255700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f168c0b6c80 0x7f168c064f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:28.534 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.525+0000 7f1692255700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168c143f90 con 0x7f168c0b31f0 2020-11-20T13:56:28.534 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.525+0000 7f1692255700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168c064580 con 0x7f168c0b6c80 2020-11-20T13:56:28.534 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.525+0000 7f1692255700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f168c064820 con 0x7f168c0b1450 2020-11-20T13:56:28.534 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.525+0000 7f168affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f168c0b31f0 0x7f168c124130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:28.535 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.525+0000 7f168affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f168c0b31f0 0x7f168c124130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54584/0 (socket says 172.21.15.40:54584) 2020-11-20T13:56:28.535 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.525+0000 7f168affd700 1 -- 172.21.15.40:0/2196554186 learned_addr learned my addr 172.21.15.40:0/2196554186 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:28.535 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.525+0000 7f168bfff700 1 --2- 172.21.15.40:0/2196554186 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f168c0b6c80 0x7f168c064f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:28.535 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.525+0000 7f168b7fe700 1 --2- 172.21.15.40:0/2196554186 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f168c0b1450 0x7f168c143650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:28.535 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.525+0000 7f168affd700 1 -- 172.21.15.40:0/2196554186 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f168c0b1450 msgr2=0x7f168c143650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:28.536 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.525+0000 7f168affd700 1 --2- 172.21.15.40:0/2196554186 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f168c0b1450 0x7f168c143650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.536 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.525+0000 7f168affd700 1 -- 172.21.15.40:0/2196554186 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f168c0b6c80 msgr2=0x7f168c064f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:28.536 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.525+0000 7f168affd700 1 --2- 172.21.15.40:0/2196554186 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f168c0b6c80 0x7f168c064f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.536 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.525+0000 7f168affd700 1 -- 172.21.15.40:0/2196554186 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f168c064ac0 con 0x7f168c0b31f0 2020-11-20T13:56:28.536 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.525+0000 7f168b7fe700 1 --2- 172.21.15.40:0/2196554186 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f168c0b1450 0x7f168c143650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:28.537 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.525+0000 7f168bfff700 1 --2- 172.21.15.40:0/2196554186 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f168c0b6c80 0x7f168c064f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:28.537 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.525+0000 7f168affd700 1 --2- 172.21.15.40:0/2196554186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f168c0b31f0 0x7f168c124130 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f168000c000 tx=0x7f1680008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:28.537 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.525+0000 7f1688ff9700 1 -- 172.21.15.40:0/2196554186 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f168000cb50 con 0x7f168c0b31f0 2020-11-20T13:56:28.537 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.525+0000 7f1688ff9700 1 -- 172.21.15.40:0/2196554186 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f168000ccb0 con 0x7f168c0b31f0 2020-11-20T13:56:28.537 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.525+0000 7f1688ff9700 1 -- 172.21.15.40:0/2196554186 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f16800125e0 con 0x7f168c0b31f0 2020-11-20T13:56:28.538 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.525+0000 7f1692255700 1 -- 172.21.15.40:0/2196554186 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f168c065450 con 0x7f168c0b31f0 2020-11-20T13:56:28.538 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.525+0000 7f1692255700 1 -- 172.21.15.40:0/2196554186 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f168c0ae7d0 con 0x7f168c0b31f0 2020-11-20T13:56:28.538 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.526+0000 7f1692255700 1 -- 172.21.15.40:0/2196554186 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f168c064c50 con 0x7f168c0b31f0 2020-11-20T13:56:28.538 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.526+0000 7f1688ff9700 1 -- 172.21.15.40:0/2196554186 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f1680012740 con 0x7f168c0b31f0 2020-11-20T13:56:28.539 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.527+0000 7f1688ff9700 1 --2- 172.21.15.40:0/2196554186 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f166c051320 0x7f166c0537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:28.539 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.527+0000 7f1688ff9700 1 -- 172.21.15.40:0/2196554186 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f16800698e0 con 0x7f168c0b31f0 2020-11-20T13:56:28.539 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.527+0000 7f168b7fe700 1 --2- 172.21.15.40:0/2196554186 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f166c051320 0x7f166c0537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:28.539 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.527+0000 7f168b7fe700 1 --2- 172.21.15.40:0/2196554186 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f166c051320 0x7f166c0537e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f167c008000 tx=0x7f167c009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:28.540 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.530+0000 7f1688ff9700 1 -- 172.21.15.40:0/2196554186 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f1680041b00 con 0x7f168c0b31f0 2020-11-20T13:56:28.690 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.690+0000 7f1692255700 1 -- 172.21.15.40:0/2196554186 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f168c057050 con 0x7f168c0b31f0 2020-11-20T13:56:28.690 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.690+0000 7f1688ff9700 1 -- 172.21.15.40:0/2196554186 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v24) v1 ==== 110+0+10 (secure 0 0 0) 0x7f1680045120 con 0x7f168c0b31f0 2020-11-20T13:56:28.691 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2020-11-20T13:56:28.692 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.691+0000 7f1692255700 1 -- 172.21.15.40:0/2196554186 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f166c051320 msgr2=0x7f166c0537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:28.692 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.691+0000 7f1692255700 1 --2- 172.21.15.40:0/2196554186 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f166c051320 0x7f166c0537e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f167c008000 tx=0x7f167c009040).stop 2020-11-20T13:56:28.692 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.691+0000 7f1692255700 1 -- 172.21.15.40:0/2196554186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f168c0b31f0 msgr2=0x7f168c124130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:28.692 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.691+0000 7f1692255700 1 --2- 172.21.15.40:0/2196554186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f168c0b31f0 0x7f168c124130 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f168000c000 tx=0x7f1680008040).stop 2020-11-20T13:56:28.693 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.691+0000 7f1692255700 1 -- 172.21.15.40:0/2196554186 shutdown_connections 2020-11-20T13:56:28.693 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.691+0000 7f1692255700 1 --2- 172.21.15.40:0/2196554186 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f166c051320 0x7f166c0537e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.693 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.691+0000 7f1692255700 1 --2- 172.21.15.40:0/2196554186 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f168c0b1450 0x7f168c143650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.693 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.691+0000 7f1692255700 1 --2- 172.21.15.40:0/2196554186 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f168c0b6c80 0x7f168c064f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.694 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.691+0000 7f1692255700 1 --2- 172.21.15.40:0/2196554186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f168c0b31f0 0x7f168c124130 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.694 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.691+0000 7f1692255700 1 -- 172.21.15.40:0/2196554186 >> 172.21.15.40:0/2196554186 conn(0x7f168c0ba1d0 msgr2=0x7f168c0b8910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:28.695 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.691+0000 7f1692255700 1 -- 172.21.15.40:0/2196554186 shutdown_connections 2020-11-20T13:56:28.695 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.691+0000 7f1692255700 1 -- 172.21.15.40:0/2196554186 wait complete. 2020-11-20T13:56:28.702 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-20T13:56:28.818 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.818+0000 7f0ebaf19700 1 Processor -- start 2020-11-20T13:56:28.819 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.818+0000 7f0ebaf19700 1 -- start start 2020-11-20T13:56:28.821 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.819+0000 7f0ebaf19700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0eb40b6c80 0x7f0eb41474c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:28.821 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.819+0000 7f0ebaf19700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0eb414ab00 0x7f0eb41410b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:28.822 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.819+0000 7f0ebaf19700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eb4147cb0 con 0x7f0eb41415a0 2020-11-20T13:56:28.822 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.819+0000 7f0ebaf19700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eb4147f50 con 0x7f0eb40b6c80 2020-11-20T13:56:28.822 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.819+0000 7f0ebaf19700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eb41481f0 con 0x7f0eb414ab00 2020-11-20T13:56:28.822 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.819+0000 7f0eb8cb5700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0eb40b6c80 0x7f0eb41474c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:28.823 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.819+0000 7f0eb94b6700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0eb41415a0 0x7f0eb405f390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45186/0 (socket says 172.21.15.40:45186) 2020-11-20T13:56:28.823 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.819+0000 7f0eb94b6700 1 -- 172.21.15.40:0/761442815 learned_addr learned my addr 172.21.15.40:0/761442815 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:28.824 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.819+0000 7f0eb3fff700 1 --2- 172.21.15.40:0/761442815 >> v2:172.21.15.116:3300/0 conn(0x7f0eb414ab00 0x7f0eb41410b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:28.824 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.819+0000 7f0eb8cb5700 1 -- 172.21.15.40:0/761442815 >> v2:172.21.15.116:3300/0 conn(0x7f0eb414ab00 msgr2=0x7f0eb41410b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:28.824 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.819+0000 7f0eb8cb5700 1 --2- 172.21.15.40:0/761442815 >> v2:172.21.15.116:3300/0 conn(0x7f0eb414ab00 0x7f0eb41410b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.824 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.819+0000 7f0eb8cb5700 1 -- 172.21.15.40:0/761442815 >> v1:172.21.15.40:6789/0 conn(0x7f0eb41415a0 legacy=0x7f0eb405f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:28.825 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.820+0000 7f0eb8cb5700 1 -- 172.21.15.40:0/761442815 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb40b7060 con 0x7f0eb40b6c80 2020-11-20T13:56:28.825 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.819+0000 7f0eb37fe700 1 -- 172.21.15.40:0/761442815 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2883507327 0 0) 0x7f0eb4147cb0 con 0x7f0eb41415a0 2020-11-20T13:56:28.825 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.820+0000 7f0eb8cb5700 1 --2- 172.21.15.40:0/761442815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0eb40b6c80 0x7f0eb41474c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f0ea8002f30 tx=0x7f0ea8006d60).ready entity=mon.0 client_cookie=db3fbd71fe618a56 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:28.825 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.820+0000 7f0eb37fe700 1 -- 172.21.15.40:0/761442815 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ea80097f0 con 0x7f0eb40b6c80 2020-11-20T13:56:28.825 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.820+0000 7f0eb37fe700 1 -- 172.21.15.40:0/761442815 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea8009950 con 0x7f0eb40b6c80 2020-11-20T13:56:28.826 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.820+0000 7f0eb37fe700 1 -- 172.21.15.40:0/761442815 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ea8009ac0 con 0x7f0eb40b6c80 2020-11-20T13:56:28.826 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.820+0000 7f0ebaf19700 1 -- 172.21.15.40:0/761442815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0eb40b6c80 msgr2=0x7f0eb41474c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:28.826 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.820+0000 7f0ebaf19700 1 --2- 172.21.15.40:0/761442815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0eb40b6c80 0x7f0eb41474c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f0ea8002f30 tx=0x7f0ea8006d60).stop 2020-11-20T13:56:28.826 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.820+0000 7f0ebaf19700 1 -- 172.21.15.40:0/761442815 shutdown_connections 2020-11-20T13:56:28.826 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.820+0000 7f0ebaf19700 1 --2- 172.21.15.40:0/761442815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0eb40b6c80 0x7f0eb41474c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.827 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.820+0000 7f0ebaf19700 1 --2- 172.21.15.40:0/761442815 >> v2:172.21.15.116:3300/0 conn(0x7f0eb414ab00 0x7f0eb41410b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.827 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.820+0000 7f0ebaf19700 1 -- 172.21.15.40:0/761442815 >> 172.21.15.40:0/761442815 conn(0x7f0eb40ba1d0 msgr2=0x7f0eb40b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:28.827 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.820+0000 7f0ebaf19700 1 -- 172.21.15.40:0/761442815 shutdown_connections 2020-11-20T13:56:28.827 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.820+0000 7f0ebaf19700 1 -- 172.21.15.40:0/761442815 wait complete. 2020-11-20T13:56:28.827 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.821+0000 7f0ebaf19700 1 Processor -- start 2020-11-20T13:56:28.828 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.821+0000 7f0ebaf19700 1 -- start start 2020-11-20T13:56:28.830 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.822+0000 7f0ebaf19700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0eb40b6c80 0x7f0eb4146500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:28.830 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.822+0000 7f0ebaf19700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0eb41415a0 0x7f0eb40ae860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:28.831 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.822+0000 7f0ebaf19700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0eb414ab00 0x7f0eb41422b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:28.831 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.822+0000 7f0ebaf19700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb40af100 con 0x7f0eb414ab00 2020-11-20T13:56:28.831 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.822+0000 7f0ebaf19700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb41437c0 con 0x7f0eb41415a0 2020-11-20T13:56:28.832 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.822+0000 7f0ebaf19700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb40aed50 con 0x7f0eb40b6c80 2020-11-20T13:56:28.832 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.822+0000 7f0eb94b6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0eb414ab00 0x7f0eb41422b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:28.832 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.822+0000 7f0eb94b6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0eb414ab00 0x7f0eb41422b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54598/0 (socket says 172.21.15.40:54598) 2020-11-20T13:56:28.832 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.822+0000 7f0eb94b6700 1 -- 172.21.15.40:0/7273283 learned_addr learned my addr 172.21.15.40:0/7273283 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:28.832 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.822+0000 7f0eb8cb5700 1 --2- 172.21.15.40:0/7273283 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0eb40b6c80 0x7f0eb4146500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:28.833 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.822+0000 7f0eb3fff700 1 --2- 172.21.15.40:0/7273283 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0eb41415a0 0x7f0eb40ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:28.833 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.823+0000 7f0eb94b6700 1 -- 172.21.15.40:0/7273283 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0eb40b6c80 msgr2=0x7f0eb4146500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:28.833 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.823+0000 7f0eb94b6700 1 --2- 172.21.15.40:0/7273283 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0eb40b6c80 0x7f0eb4146500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.833 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.823+0000 7f0eb94b6700 1 -- 172.21.15.40:0/7273283 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0eb41415a0 msgr2=0x7f0eb40ae860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:28.833 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.823+0000 7f0eb94b6700 1 --2- 172.21.15.40:0/7273283 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0eb41415a0 0x7f0eb40ae860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.834 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.823+0000 7f0eb94b6700 1 -- 172.21.15.40:0/7273283 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb41427a0 con 0x7f0eb414ab00 2020-11-20T13:56:28.834 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.823+0000 7f0eb94b6700 1 --2- 172.21.15.40:0/7273283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0eb414ab00 0x7f0eb41422b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f0eac000f00 tx=0x7f0eac007cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:28.834 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.823+0000 7f0eb1ffb700 1 -- 172.21.15.40:0/7273283 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0eac005960 con 0x7f0eb414ab00 2020-11-20T13:56:28.834 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.823+0000 7f0eb1ffb700 1 -- 172.21.15.40:0/7273283 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eac005ac0 con 0x7f0eb414ab00 2020-11-20T13:56:28.835 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.823+0000 7f0ebaf19700 1 -- 172.21.15.40:0/7273283 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eb41429f0 con 0x7f0eb414ab00 2020-11-20T13:56:28.835 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.823+0000 7f0eb1ffb700 1 -- 172.21.15.40:0/7273283 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0eac005e20 con 0x7f0eb414ab00 2020-11-20T13:56:28.835 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.823+0000 7f0ebaf19700 1 -- 172.21.15.40:0/7273283 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eb4143ea0 con 0x7f0eb414ab00 2020-11-20T13:56:28.835 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.824+0000 7f0ebaf19700 1 -- 172.21.15.40:0/7273283 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eb4143600 con 0x7f0eb414ab00 2020-11-20T13:56:28.835 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.824+0000 7f0eb1ffb700 1 -- 172.21.15.40:0/7273283 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f0eac010030 con 0x7f0eb414ab00 2020-11-20T13:56:28.836 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.824+0000 7f0eb1ffb700 1 --2- 172.21.15.40:0/7273283 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f0e9c051210 0x7f0e9c0536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:28.836 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.825+0000 7f0eb1ffb700 1 -- 172.21.15.40:0/7273283 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f0eac068ea0 con 0x7f0eb414ab00 2020-11-20T13:56:28.836 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.825+0000 7f0eb8cb5700 1 --2- 172.21.15.40:0/7273283 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f0e9c051210 0x7f0e9c0536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:28.836 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.825+0000 7f0eb8cb5700 1 --2- 172.21.15.40:0/7273283 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f0e9c051210 0x7f0e9c0536d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f0ea801b000 tx=0x7f0ea8016000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:28.836 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.828+0000 7f0eb1ffb700 1 -- 172.21.15.40:0/7273283 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f0eac0410c0 con 0x7f0eb414ab00 2020-11-20T13:56:28.994 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.994+0000 7f0ebaf19700 1 -- 172.21.15.40:0/7273283 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0eb4143980 con 0x7f0eb414ab00 2020-11-20T13:56:28.994 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.994+0000 7f0eb1ffb700 1 -- 172.21.15.40:0/7273283 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0eac0446e0 con 0x7f0eb414ab00 2020-11-20T13:56:28.994 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2020-11-20T13:56:28.996 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.995+0000 7f0ebaf19700 1 -- 172.21.15.40:0/7273283 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f0e9c051210 msgr2=0x7f0e9c0536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:28.996 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.995+0000 7f0ebaf19700 1 --2- 172.21.15.40:0/7273283 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f0e9c051210 0x7f0e9c0536d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f0ea801b000 tx=0x7f0ea8016000).stop 2020-11-20T13:56:28.996 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.995+0000 7f0ebaf19700 1 -- 172.21.15.40:0/7273283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0eb414ab00 msgr2=0x7f0eb41422b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:28.996 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.995+0000 7f0ebaf19700 1 --2- 172.21.15.40:0/7273283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0eb414ab00 0x7f0eb41422b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f0eac000f00 tx=0x7f0eac007cb0).stop 2020-11-20T13:56:28.997 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.996+0000 7f0ebaf19700 1 -- 172.21.15.40:0/7273283 shutdown_connections 2020-11-20T13:56:28.997 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.996+0000 7f0ebaf19700 1 --2- 172.21.15.40:0/7273283 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f0e9c051210 0x7f0e9c0536d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.997 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.996+0000 7f0ebaf19700 1 --2- 172.21.15.40:0/7273283 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0eb40b6c80 0x7f0eb4146500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.997 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.996+0000 7f0ebaf19700 1 --2- 172.21.15.40:0/7273283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0eb414ab00 0x7f0eb41422b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.998 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.996+0000 7f0ebaf19700 1 --2- 172.21.15.40:0/7273283 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0eb41415a0 0x7f0eb40ae860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.998 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.996+0000 7f0ebaf19700 1 -- 172.21.15.40:0/7273283 >> 172.21.15.40:0/7273283 conn(0x7f0eb40ba1d0 msgr2=0x7f0eb4145070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:28.998 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.996+0000 7f0ebaf19700 1 -- 172.21.15.40:0/7273283 shutdown_connections 2020-11-20T13:56:28.998 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:28.996+0000 7f0ebaf19700 1 -- 172.21.15.40:0/7273283 wait complete. 2020-11-20T13:56:29.006 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-20T13:56:29.124 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.124+0000 7faa658b9700 1 Processor -- start 2020-11-20T13:56:29.125 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.124+0000 7faa658b9700 1 -- start start 2020-11-20T13:56:29.127 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.125+0000 7faa658b9700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7faa60061f90 0x7faa6014b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:29.127 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.125+0000 7faa658b9700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7faa600b6980 0x7faa600b6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:29.127 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.125+0000 7faa658b9700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa60061310 con 0x7faa6005c510 2020-11-20T13:56:29.127 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.125+0000 7faa658b9700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa600615b0 con 0x7faa600b6980 2020-11-20T13:56:29.128 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.125+0000 7faa658b9700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa60061850 con 0x7faa60061f90 2020-11-20T13:56:29.128 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.125+0000 7faa5f7fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7faa600b6980 0x7faa600b6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:29.128 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.125+0000 7faa5f7fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7faa600b6980 0x7faa600b6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54602/0 (socket says 172.21.15.40:54602) 2020-11-20T13:56:29.128 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.125+0000 7faa5f7fe700 1 -- 172.21.15.40:0/2084530816 learned_addr learned my addr 172.21.15.40:0/2084530816 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:29.128 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.125+0000 7faa5e7fc700 1 --2- 172.21.15.40:0/2084530816 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7faa60061f90 0x7faa6014b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:29.129 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.125+0000 7faa5f7fe700 1 -- 172.21.15.40:0/2084530816 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7faa60061f90 msgr2=0x7faa6014b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:29.129 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.126+0000 7faa5f7fe700 1 --2- 172.21.15.40:0/2084530816 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7faa60061f90 0x7faa6014b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.129 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.126+0000 7faa5f7fe700 1 -- 172.21.15.40:0/2084530816 >> v1:172.21.15.116:6789/0 conn(0x7faa6005c510 legacy=0x7faa6012d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:29.129 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.126+0000 7faa5f7fe700 1 -- 172.21.15.40:0/2084530816 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa60061af0 con 0x7faa600b6980 2020-11-20T13:56:29.130 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.126+0000 7faa5f7fe700 1 --2- 172.21.15.40:0/2084530816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa600b6980 0x7faa600b6df0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7faa60062370 tx=0x7faa58009270).ready entity=mon.0 client_cookie=68fba6ad9f0aaa95 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:29.130 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.126+0000 7faa5dffb700 1 -- 172.21.15.40:0/2084530816 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa5800bbc0 con 0x7faa600b6980 2020-11-20T13:56:29.130 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.126+0000 7faa5dffb700 1 -- 172.21.15.40:0/2084530816 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa5800bd20 con 0x7faa600b6980 2020-11-20T13:56:29.130 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.126+0000 7faa5dffb700 1 -- 172.21.15.40:0/2084530816 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa58011400 con 0x7faa600b6980 2020-11-20T13:56:29.131 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.126+0000 7faa658b9700 1 -- 172.21.15.40:0/2084530816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa600b6980 msgr2=0x7faa600b6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:29.131 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.126+0000 7faa658b9700 1 --2- 172.21.15.40:0/2084530816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa600b6980 0x7faa600b6df0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7faa60062370 tx=0x7faa58009270).stop 2020-11-20T13:56:29.131 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.126+0000 7faa658b9700 1 -- 172.21.15.40:0/2084530816 shutdown_connections 2020-11-20T13:56:29.131 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.126+0000 7faa658b9700 1 --2- 172.21.15.40:0/2084530816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa600b6980 0x7faa600b6df0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.132 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.126+0000 7faa658b9700 1 --2- 172.21.15.40:0/2084530816 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7faa60061f90 0x7faa6014b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.132 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.126+0000 7faa658b9700 1 -- 172.21.15.40:0/2084530816 >> 172.21.15.40:0/2084530816 conn(0x7faa600bdb70 msgr2=0x7faa600bdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:29.132 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.126+0000 7faa658b9700 1 -- 172.21.15.40:0/2084530816 shutdown_connections 2020-11-20T13:56:29.132 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.126+0000 7faa658b9700 1 -- 172.21.15.40:0/2084530816 wait complete. 2020-11-20T13:56:29.135 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.127+0000 7faa658b9700 1 Processor -- start 2020-11-20T13:56:29.135 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.127+0000 7faa658b9700 1 -- start start 2020-11-20T13:56:29.135 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.128+0000 7faa658b9700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7faa6005c510 0x7faa6014a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:29.135 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.128+0000 7faa658b9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa60061f90 0x7faa601478a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:29.136 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.128+0000 7faa658b9700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa600b6980 0x7faa600b0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:29.136 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.128+0000 7faa658b9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa601571c0 con 0x7faa60061f90 2020-11-20T13:56:29.136 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.128+0000 7faa658b9700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa60159b40 con 0x7faa600b6980 2020-11-20T13:56:29.136 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.128+0000 7faa658b9700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa60156e30 con 0x7faa6005c510 2020-11-20T13:56:29.136 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.128+0000 7faa5e7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa60061f90 0x7faa601478a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:29.137 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.128+0000 7faa5e7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa60061f90 0x7faa601478a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54608/0 (socket says 172.21.15.40:54608) 2020-11-20T13:56:29.137 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.128+0000 7faa5e7fc700 1 -- 172.21.15.40:0/2728766707 learned_addr learned my addr 172.21.15.40:0/2728766707 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:29.137 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.128+0000 7faa5effd700 1 --2- 172.21.15.40:0/2728766707 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7faa6005c510 0x7faa6014a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:29.137 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.128+0000 7faa5f7fe700 1 --2- 172.21.15.40:0/2728766707 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa600b6980 0x7faa600b0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:29.137 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.128+0000 7faa5e7fc700 1 -- 172.21.15.40:0/2728766707 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7faa6005c510 msgr2=0x7faa6014a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:29.138 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.128+0000 7faa5e7fc700 1 --2- 172.21.15.40:0/2728766707 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7faa6005c510 0x7faa6014a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.138 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.128+0000 7faa5e7fc700 1 -- 172.21.15.40:0/2728766707 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa600b6980 msgr2=0x7faa600b0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:29.138 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.128+0000 7faa5e7fc700 1 --2- 172.21.15.40:0/2728766707 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa600b6980 0x7faa600b0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.138 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.128+0000 7faa5e7fc700 1 -- 172.21.15.40:0/2728766707 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa600ae870 con 0x7faa60061f90 2020-11-20T13:56:29.138 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.129+0000 7faa5e7fc700 1 --2- 172.21.15.40:0/2728766707 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa60061f90 0x7faa601478a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7faa50003070 tx=0x7faa50008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:29.139 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.129+0000 7faa4ffff700 1 -- 172.21.15.40:0/2728766707 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa50009790 con 0x7faa60061f90 2020-11-20T13:56:29.139 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.129+0000 7faa4ffff700 1 -- 172.21.15.40:0/2728766707 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa500098f0 con 0x7faa60061f90 2020-11-20T13:56:29.139 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.129+0000 7faa4ffff700 1 -- 172.21.15.40:0/2728766707 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa50009c50 con 0x7faa60061f90 2020-11-20T13:56:29.139 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.129+0000 7faa658b9700 1 -- 172.21.15.40:0/2728766707 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa600b0760 con 0x7faa60061f90 2020-11-20T13:56:29.139 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.129+0000 7faa658b9700 1 -- 172.21.15.40:0/2728766707 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa60143110 con 0x7faa60061f90 2020-11-20T13:56:29.140 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.130+0000 7faa658b9700 1 -- 172.21.15.40:0/2728766707 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa600aea60 con 0x7faa60061f90 2020-11-20T13:56:29.140 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.130+0000 7faa4ffff700 1 -- 172.21.15.40:0/2728766707 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7faa50013480 con 0x7faa60061f90 2020-11-20T13:56:29.140 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.130+0000 7faa4ffff700 1 --2- 172.21.15.40:0/2728766707 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7faa3c051290 0x7faa3c053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:29.140 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.130+0000 7faa4ffff700 1 -- 172.21.15.40:0/2728766707 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7faa50069500 con 0x7faa60061f90 2020-11-20T13:56:29.140 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.130+0000 7faa5effd700 1 --2- 172.21.15.40:0/2728766707 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7faa3c051290 0x7faa3c053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:29.141 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.131+0000 7faa5effd700 1 --2- 172.21.15.40:0/2728766707 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7faa3c051290 0x7faa3c053750 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7faa5400d000 tx=0x7faa54007ab0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:29.141 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.134+0000 7faa4ffff700 1 -- 172.21.15.40:0/2728766707 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7faa600aea60 con 0x7faa60061f90 2020-11-20T13:56:29.292 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.292+0000 7faa658b9700 1 -- 172.21.15.40:0/2728766707 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faa600aea60 con 0x7faa60061f90 2020-11-20T13:56:29.292 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.292+0000 7faa4ffff700 1 -- 172.21.15.40:0/2728766707 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+10 (secure 0 0 0) 0x7faa600aea60 con 0x7faa60061f90 2020-11-20T13:56:29.293 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2020-11-20T13:56:29.294 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.293+0000 7faa658b9700 1 -- 172.21.15.40:0/2728766707 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7faa3c051290 msgr2=0x7faa3c053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:29.294 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.293+0000 7faa658b9700 1 --2- 172.21.15.40:0/2728766707 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7faa3c051290 0x7faa3c053750 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7faa5400d000 tx=0x7faa54007ab0).stop 2020-11-20T13:56:29.294 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.293+0000 7faa658b9700 1 -- 172.21.15.40:0/2728766707 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa60061f90 msgr2=0x7faa601478a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:29.294 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.293+0000 7faa658b9700 1 --2- 172.21.15.40:0/2728766707 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa60061f90 0x7faa601478a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7faa50003070 tx=0x7faa50008040).stop 2020-11-20T13:56:29.295 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.294+0000 7faa658b9700 1 -- 172.21.15.40:0/2728766707 shutdown_connections 2020-11-20T13:56:29.295 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.294+0000 7faa658b9700 1 --2- 172.21.15.40:0/2728766707 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7faa3c051290 0x7faa3c053750 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.295 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.294+0000 7faa658b9700 1 --2- 172.21.15.40:0/2728766707 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7faa6005c510 0x7faa6014a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.295 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.294+0000 7faa658b9700 1 --2- 172.21.15.40:0/2728766707 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa600b6980 0x7faa600b0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.296 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.294+0000 7faa658b9700 1 --2- 172.21.15.40:0/2728766707 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa60061f90 0x7faa601478a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.296 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.294+0000 7faa658b9700 1 -- 172.21.15.40:0/2728766707 >> 172.21.15.40:0/2728766707 conn(0x7faa600bdb70 msgr2=0x7faa60144cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:29.296 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.294+0000 7faa658b9700 1 -- 172.21.15.40:0/2728766707 shutdown_connections 2020-11-20T13:56:29.297 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.294+0000 7faa658b9700 1 -- 172.21.15.40:0/2728766707 wait complete. 2020-11-20T13:56:29.304 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-20T13:56:29.423 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.420+0000 7fb631d2e700 1 Processor -- start 2020-11-20T13:56:29.424 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.420+0000 7fb631d2e700 1 -- start start 2020-11-20T13:56:29.425 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.421+0000 7fb631d2e700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb62c0b31f0 0x7fb62c144bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:29.425 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.421+0000 7fb631d2e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb62c0b1450 0x7fb62c0b1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:29.425 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.421+0000 7fb631d2e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb62c0b1df0 con 0x7fb62c0b6c80 2020-11-20T13:56:29.425 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.421+0000 7fb631d2e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb62c0b0d10 con 0x7fb62c0b1450 2020-11-20T13:56:29.426 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.421+0000 7fb631d2e700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb62c0b0fb0 con 0x7fb62c0b31f0 2020-11-20T13:56:29.426 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.421+0000 7fb62b7fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb62c0b6c80 0x7fb62c0b40f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45212/0 (socket says 172.21.15.40:45212) 2020-11-20T13:56:29.427 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.421+0000 7fb62b7fe700 1 -- 172.21.15.40:0/1650362027 learned_addr learned my addr 172.21.15.40:0/1650362027 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:29.427 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.421+0000 7fb62bfff700 1 --2- 172.21.15.40:0/1650362027 >> v2:172.21.15.40:3300/0 conn(0x7fb62c0b1450 0x7fb62c0b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:29.427 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.421+0000 7fb62affd700 1 --2- 172.21.15.40:0/1650362027 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb62c0b31f0 0x7fb62c144bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:29.427 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.422+0000 7fb62a7fc700 1 -- 172.21.15.40:0/1650362027 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1081920724 0 0) 0x7fb62c0b1df0 con 0x7fb62c0b6c80 2020-11-20T13:56:29.427 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.422+0000 7fb62a7fc700 1 -- 172.21.15.40:0/1650362027 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb618003230 con 0x7fb62c0b6c80 2020-11-20T13:56:29.428 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.422+0000 7fb62bfff700 1 -- 172.21.15.40:0/1650362027 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb62c0b31f0 msgr2=0x7fb62c144bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:29.428 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.422+0000 7fb62bfff700 1 --2- 172.21.15.40:0/1650362027 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb62c0b31f0 0x7fb62c144bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.428 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.422+0000 7fb62bfff700 1 -- 172.21.15.40:0/1650362027 >> v1:172.21.15.40:6789/0 conn(0x7fb62c0b6c80 legacy=0x7fb62c0b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:29.428 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.422+0000 7fb62bfff700 1 -- 172.21.15.40:0/1650362027 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb62c0b7060 con 0x7fb62c0b1450 2020-11-20T13:56:29.428 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.422+0000 7fb62bfff700 1 --2- 172.21.15.40:0/1650362027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb62c0b1450 0x7fb62c0b1900 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fb62c0b3100 tx=0x7fb6240091a0).ready entity=mon.0 client_cookie=530429a420845128 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:29.429 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.422+0000 7fb62a7fc700 1 -- 172.21.15.40:0/1650362027 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb62400b900 con 0x7fb62c0b1450 2020-11-20T13:56:29.429 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.422+0000 7fb62a7fc700 1 -- 172.21.15.40:0/1650362027 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb62400ba60 con 0x7fb62c0b1450 2020-11-20T13:56:29.429 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.422+0000 7fb62a7fc700 1 -- 172.21.15.40:0/1650362027 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb62400bbd0 con 0x7fb62c0b1450 2020-11-20T13:56:29.429 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.422+0000 7fb631d2e700 1 -- 172.21.15.40:0/1650362027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb62c0b1450 msgr2=0x7fb62c0b1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:29.429 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.422+0000 7fb631d2e700 1 --2- 172.21.15.40:0/1650362027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb62c0b1450 0x7fb62c0b1900 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fb62c0b3100 tx=0x7fb6240091a0).stop 2020-11-20T13:56:29.430 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.422+0000 7fb631d2e700 1 -- 172.21.15.40:0/1650362027 shutdown_connections 2020-11-20T13:56:29.430 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.422+0000 7fb631d2e700 1 --2- 172.21.15.40:0/1650362027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb62c0b1450 0x7fb62c0b1900 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.430 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.422+0000 7fb631d2e700 1 --2- 172.21.15.40:0/1650362027 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb62c0b31f0 0x7fb62c144bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.431 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.422+0000 7fb631d2e700 1 -- 172.21.15.40:0/1650362027 >> 172.21.15.40:0/1650362027 conn(0x7fb62c0ba1d0 msgr2=0x7fb62c0b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:29.431 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.423+0000 7fb631d2e700 1 -- 172.21.15.40:0/1650362027 shutdown_connections 2020-11-20T13:56:29.431 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.423+0000 7fb631d2e700 1 -- 172.21.15.40:0/1650362027 wait complete. 2020-11-20T13:56:29.431 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.423+0000 7fb631d2e700 1 Processor -- start 2020-11-20T13:56:29.434 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.424+0000 7fb631d2e700 1 -- start start 2020-11-20T13:56:29.435 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.424+0000 7fb631d2e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb62c0b1450 0x7fb62c156240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:29.436 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.424+0000 7fb631d2e700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb62c0b31f0 0x7fb62c05f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:29.436 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.424+0000 7fb631d2e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb62c0b6c80 0x7fb62c159f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:29.436 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.424+0000 7fb631d2e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62c12d5a0 con 0x7fb62c0b6c80 2020-11-20T13:56:29.437 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.424+0000 7fb631d2e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62c156870 con 0x7fb62c0b1450 2020-11-20T13:56:29.437 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.424+0000 7fb631d2e700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb62c156b10 con 0x7fb62c0b31f0 2020-11-20T13:56:29.437 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.424+0000 7fb62bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb62c0b6c80 0x7fb62c159f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:29.437 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.424+0000 7fb62bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb62c0b6c80 0x7fb62c159f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54626/0 (socket says 172.21.15.40:54626) 2020-11-20T13:56:29.438 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.424+0000 7fb62bfff700 1 -- 172.21.15.40:0/2897675339 learned_addr learned my addr 172.21.15.40:0/2897675339 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:29.438 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.425+0000 7fb62b7fe700 1 --2- 172.21.15.40:0/2897675339 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb62c0b1450 0x7fb62c156240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:29.438 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.425+0000 7fb62affd700 1 --2- 172.21.15.40:0/2897675339 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb62c0b31f0 0x7fb62c05f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:29.438 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.425+0000 7fb62bfff700 1 -- 172.21.15.40:0/2897675339 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb62c0b31f0 msgr2=0x7fb62c05f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:29.439 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.425+0000 7fb62bfff700 1 --2- 172.21.15.40:0/2897675339 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb62c0b31f0 0x7fb62c05f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.439 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.425+0000 7fb62bfff700 1 -- 172.21.15.40:0/2897675339 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb62c0b1450 msgr2=0x7fb62c156240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:29.439 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.425+0000 7fb62bfff700 1 --2- 172.21.15.40:0/2897675339 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb62c0b1450 0x7fb62c156240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.440 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.425+0000 7fb62bfff700 1 -- 172.21.15.40:0/2897675339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb62c156db0 con 0x7fb62c0b6c80 2020-11-20T13:56:29.440 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.425+0000 7fb62b7fe700 1 --2- 172.21.15.40:0/2897675339 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb62c0b1450 0x7fb62c156240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:29.440 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.425+0000 7fb62affd700 1 --2- 172.21.15.40:0/2897675339 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb62c0b31f0 0x7fb62c05f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:29.440 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.425+0000 7fb62bfff700 1 --2- 172.21.15.40:0/2897675339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb62c0b6c80 0x7fb62c159f30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fb62c0b3100 tx=0x7fb624016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:29.440 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.425+0000 7fb628ff9700 1 -- 172.21.15.40:0/2897675339 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb62401f3f0 con 0x7fb62c0b6c80 2020-11-20T13:56:29.441 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.425+0000 7fb628ff9700 1 -- 172.21.15.40:0/2897675339 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb62401f550 con 0x7fb62c0b6c80 2020-11-20T13:56:29.441 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.425+0000 7fb631d2e700 1 -- 172.21.15.40:0/2897675339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb62c159850 con 0x7fb62c0b6c80 2020-11-20T13:56:29.441 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.425+0000 7fb628ff9700 1 -- 172.21.15.40:0/2897675339 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb62401f8b0 con 0x7fb62c0b6c80 2020-11-20T13:56:29.441 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.425+0000 7fb631d2e700 1 -- 172.21.15.40:0/2897675339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb62c15a420 con 0x7fb62c0b6c80 2020-11-20T13:56:29.442 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.426+0000 7fb631d2e700 1 -- 172.21.15.40:0/2897675339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb62c157170 con 0x7fb62c0b6c80 2020-11-20T13:56:29.442 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.426+0000 7fb628ff9700 1 -- 172.21.15.40:0/2897675339 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fb624026030 con 0x7fb62c0b6c80 2020-11-20T13:56:29.442 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.427+0000 7fb628ff9700 1 --2- 172.21.15.40:0/2897675339 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fb60c0513c0 0x7fb60c053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:29.442 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.427+0000 7fb628ff9700 1 -- 172.21.15.40:0/2897675339 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fb624069c90 con 0x7fb62c0b6c80 2020-11-20T13:56:29.443 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.427+0000 7fb62b7fe700 1 --2- 172.21.15.40:0/2897675339 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fb60c0513c0 0x7fb60c053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:29.443 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.427+0000 7fb62b7fe700 1 --2- 172.21.15.40:0/2897675339 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fb60c0513c0 0x7fb60c053880 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fb61c008000 tx=0x7fb61c009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:29.443 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.430+0000 7fb628ff9700 1 -- 172.21.15.40:0/2897675339 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fb624041ef0 con 0x7fb62c0b6c80 2020-11-20T13:56:29.593 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.592+0000 7fb631d2e700 1 -- 172.21.15.40:0/2897675339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb62c156f40 con 0x7fb62c0b6c80 2020-11-20T13:56:29.594 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.592+0000 7fb628ff9700 1 -- 172.21.15.40:0/2897675339 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+10 (secure 0 0 0) 0x7fb6240240a0 con 0x7fb62c0b6c80 2020-11-20T13:56:29.594 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2020-11-20T13:56:29.595 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.594+0000 7fb631d2e700 1 -- 172.21.15.40:0/2897675339 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fb60c0513c0 msgr2=0x7fb60c053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:29.595 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.594+0000 7fb631d2e700 1 --2- 172.21.15.40:0/2897675339 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fb60c0513c0 0x7fb60c053880 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fb61c008000 tx=0x7fb61c009040).stop 2020-11-20T13:56:29.595 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.594+0000 7fb631d2e700 1 -- 172.21.15.40:0/2897675339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb62c0b6c80 msgr2=0x7fb62c159f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:29.596 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.594+0000 7fb631d2e700 1 --2- 172.21.15.40:0/2897675339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb62c0b6c80 0x7fb62c159f30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fb62c0b3100 tx=0x7fb624016000).stop 2020-11-20T13:56:29.596 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.594+0000 7fb631d2e700 1 -- 172.21.15.40:0/2897675339 shutdown_connections 2020-11-20T13:56:29.597 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.594+0000 7fb631d2e700 1 --2- 172.21.15.40:0/2897675339 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fb60c0513c0 0x7fb60c053880 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.597 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.594+0000 7fb631d2e700 1 --2- 172.21.15.40:0/2897675339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb62c0b6c80 0x7fb62c159f30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.597 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.594+0000 7fb631d2e700 1 --2- 172.21.15.40:0/2897675339 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb62c0b1450 0x7fb62c156240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.598 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.594+0000 7fb631d2e700 1 --2- 172.21.15.40:0/2897675339 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb62c0b31f0 0x7fb62c05f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.598 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.594+0000 7fb631d2e700 1 -- 172.21.15.40:0/2897675339 >> 172.21.15.40:0/2897675339 conn(0x7fb62c0ba1d0 msgr2=0x7fb62c144800 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:29.598 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.594+0000 7fb631d2e700 1 -- 172.21.15.40:0/2897675339 shutdown_connections 2020-11-20T13:56:29.598 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.594+0000 7fb631d2e700 1 -- 172.21.15.40:0/2897675339 wait complete. 2020-11-20T13:56:29.604 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2020-11-20T13:56:29.605 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2020-11-20T13:56:29.605 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early mgr dump --format=json 2020-11-20T13:56:29.719 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.718+0000 7fa8867b1700 1 Processor -- start 2020-11-20T13:56:29.719 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.719+0000 7fa8867b1700 1 -- start start 2020-11-20T13:56:29.722 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.719+0000 7fa8867b1700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa880067260 0x7fa8800661a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:29.722 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.719+0000 7fa8867b1700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa880142a60 0x7fa880142ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:29.722 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.719+0000 7fa8867b1700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa8801433b0 0x7fa880146b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:29.723 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.719+0000 7fa8867b1700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa880066af0 con 0x7fa8801433b0 2020-11-20T13:56:29.723 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.719+0000 7fa8867b1700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8800639f0 con 0x7fa880067260 2020-11-20T13:56:29.723 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.719+0000 7fa8867b1700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa880063c90 con 0x7fa880142a60 2020-11-20T13:56:29.723 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.720+0000 7fa885fb0700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa8801433b0 0x7fa880146b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:29.724 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.720+0000 7fa885fb0700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa8801433b0 0x7fa880146b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54630/0 (socket says 172.21.15.40:54630) 2020-11-20T13:56:29.724 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.720+0000 7fa885fb0700 1 -- 172.21.15.40:0/2776515664 learned_addr learned my addr 172.21.15.40:0/2776515664 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:29.724 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.720+0000 7fa884fae700 1 --2- 172.21.15.40:0/2776515664 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa880142a60 0x7fa880142ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:29.724 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.720+0000 7fa8857af700 1 --2- 172.21.15.40:0/2776515664 >> v2:172.21.15.116:3300/0 conn(0x7fa880067260 0x7fa8800661a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:29.724 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.720+0000 7fa885fb0700 1 -- 172.21.15.40:0/2776515664 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa880142a60 msgr2=0x7fa880142ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:29.725 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.720+0000 7fa885fb0700 1 --2- 172.21.15.40:0/2776515664 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa880142a60 0x7fa880142ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.725 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.720+0000 7fa885fb0700 1 -- 172.21.15.40:0/2776515664 >> v2:172.21.15.116:3300/0 conn(0x7fa880067260 msgr2=0x7fa8800661a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:29.725 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.720+0000 7fa885fb0700 1 --2- 172.21.15.40:0/2776515664 >> v2:172.21.15.116:3300/0 conn(0x7fa880067260 0x7fa8800661a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.725 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.720+0000 7fa885fb0700 1 -- 172.21.15.40:0/2776515664 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa880067640 con 0x7fa8801433b0 2020-11-20T13:56:29.725 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.720+0000 7fa885fb0700 1 --2- 172.21.15.40:0/2776515664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8801433b0 0x7fa880146b80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7fa8780032b0 tx=0x7fa878009270).ready entity=mon.0 client_cookie=a7a2fbc1dbf38393 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:29.726 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.720+0000 7fa873fff700 1 -- 172.21.15.40:0/2776515664 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa878009e60 con 0x7fa8801433b0 2020-11-20T13:56:29.726 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.720+0000 7fa873fff700 1 -- 172.21.15.40:0/2776515664 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa87800bb50 con 0x7fa8801433b0 2020-11-20T13:56:29.726 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.720+0000 7fa873fff700 1 -- 172.21.15.40:0/2776515664 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa87800bcc0 con 0x7fa8801433b0 2020-11-20T13:56:29.726 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.720+0000 7fa8867b1700 1 -- 172.21.15.40:0/2776515664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8801433b0 msgr2=0x7fa880146b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:29.726 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.720+0000 7fa8867b1700 1 --2- 172.21.15.40:0/2776515664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8801433b0 0x7fa880146b80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7fa8780032b0 tx=0x7fa878009270).stop 2020-11-20T13:56:29.727 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.721+0000 7fa8867b1700 1 -- 172.21.15.40:0/2776515664 shutdown_connections 2020-11-20T13:56:29.727 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.721+0000 7fa8867b1700 1 --2- 172.21.15.40:0/2776515664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8801433b0 0x7fa880146b80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.727 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.721+0000 7fa8867b1700 1 --2- 172.21.15.40:0/2776515664 >> v2:172.21.15.116:3300/0 conn(0x7fa880067260 0x7fa8800661a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.727 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.721+0000 7fa8867b1700 1 --2- 172.21.15.40:0/2776515664 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa880142a60 0x7fa880142ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.727 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.721+0000 7fa8867b1700 1 -- 172.21.15.40:0/2776515664 >> 172.21.15.40:0/2776515664 conn(0x7fa88005fe00 msgr2=0x7fa8800b6070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:29.728 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.721+0000 7fa8867b1700 1 -- 172.21.15.40:0/2776515664 shutdown_connections 2020-11-20T13:56:29.728 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.721+0000 7fa8867b1700 1 -- 172.21.15.40:0/2776515664 wait complete. 2020-11-20T13:56:29.728 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.722+0000 7fa8867b1700 1 Processor -- start 2020-11-20T13:56:29.728 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.722+0000 7fa8867b1700 1 -- start start 2020-11-20T13:56:29.730 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.722+0000 7fa8867b1700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa880067260 0x7fa880136d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:29.731 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.722+0000 7fa8867b1700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa880142a60 0x7fa880130b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:29.731 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.722+0000 7fa8867b1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8801433b0 0x7fa880131060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:29.731 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.722+0000 7fa8867b1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88013a210 con 0x7fa8801433b0 2020-11-20T13:56:29.731 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.722+0000 7fa8867b1700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa880131550 con 0x7fa880142a60 2020-11-20T13:56:29.732 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.722+0000 7fa8867b1700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa880141660 con 0x7fa880067260 2020-11-20T13:56:29.732 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.723+0000 7fa885fb0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8801433b0 0x7fa880131060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:29.732 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.723+0000 7fa885fb0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8801433b0 0x7fa880131060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54640/0 (socket says 172.21.15.40:54640) 2020-11-20T13:56:29.732 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.723+0000 7fa885fb0700 1 -- 172.21.15.40:0/3359991739 learned_addr learned my addr 172.21.15.40:0/3359991739 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:29.732 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.723+0000 7fa8857af700 1 --2- 172.21.15.40:0/3359991739 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa880067260 0x7fa880136d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:29.733 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.723+0000 7fa884fae700 1 --2- 172.21.15.40:0/3359991739 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa880142a60 0x7fa880130b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:29.733 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.723+0000 7fa885fb0700 1 -- 172.21.15.40:0/3359991739 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa880067260 msgr2=0x7fa880136d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:29.733 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.723+0000 7fa885fb0700 1 --2- 172.21.15.40:0/3359991739 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa880067260 0x7fa880136d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.733 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.723+0000 7fa885fb0700 1 -- 172.21.15.40:0/3359991739 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa880142a60 msgr2=0x7fa880130b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:29.733 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.723+0000 7fa885fb0700 1 --2- 172.21.15.40:0/3359991739 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa880142a60 0x7fa880130b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.734 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.723+0000 7fa885fb0700 1 -- 172.21.15.40:0/3359991739 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa880141900 con 0x7fa8801433b0 2020-11-20T13:56:29.734 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.723+0000 7fa885fb0700 1 --2- 172.21.15.40:0/3359991739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8801433b0 0x7fa880131060 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fa8780032b0 tx=0x7fa878016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:29.734 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.723+0000 7fa8727fc700 1 -- 172.21.15.40:0/3359991739 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa87801f3f0 con 0x7fa8801433b0 2020-11-20T13:56:29.734 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.723+0000 7fa8727fc700 1 -- 172.21.15.40:0/3359991739 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa87801f550 con 0x7fa8801433b0 2020-11-20T13:56:29.734 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.723+0000 7fa8867b1700 1 -- 172.21.15.40:0/3359991739 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa880137d50 con 0x7fa8801433b0 2020-11-20T13:56:29.735 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.723+0000 7fa8727fc700 1 -- 172.21.15.40:0/3359991739 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa87801f8b0 con 0x7fa8801433b0 2020-11-20T13:56:29.735 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.723+0000 7fa8867b1700 1 -- 172.21.15.40:0/3359991739 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa880138a40 con 0x7fa8801433b0 2020-11-20T13:56:29.735 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.724+0000 7fa8867b1700 1 -- 172.21.15.40:0/3359991739 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa880057050 con 0x7fa8801433b0 2020-11-20T13:56:29.735 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.724+0000 7fa8727fc700 1 -- 172.21.15.40:0/3359991739 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fa878026030 con 0x7fa8801433b0 2020-11-20T13:56:29.735 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.725+0000 7fa8727fc700 1 --2- 172.21.15.40:0/3359991739 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa860051320 0x7fa8600537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:29.736 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.725+0000 7fa8727fc700 1 -- 172.21.15.40:0/3359991739 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fa878069ba0 con 0x7fa8801433b0 2020-11-20T13:56:29.736 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.725+0000 7fa8857af700 1 --2- 172.21.15.40:0/3359991739 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa860051320 0x7fa8600537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:29.736 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.725+0000 7fa8857af700 1 --2- 172.21.15.40:0/3359991739 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa860051320 0x7fa8600537e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fa87400b000 tx=0x7fa874006dc0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:29.736 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.728+0000 7fa8727fc700 1 -- 172.21.15.40:0/3359991739 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fa878041dc0 con 0x7fa8801433b0 2020-11-20T13:56:29.932 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.932+0000 7fa8867b1700 1 -- 172.21.15.40:0/3359991739 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fa8800699a0 con 0x7fa8801433b0 2020-11-20T13:56:29.935 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.935+0000 7fa8727fc700 1 -- 172.21.15.40:0/3359991739 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v4) v1 ==== 74+0+126774 (secure 0 0 0) 0x7fa878024090 con 0x7fa8801433b0 2020-11-20T13:56:29.937 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:56:29.938 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.937+0000 7fa8867b1700 1 -- 172.21.15.40:0/3359991739 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa860051320 msgr2=0x7fa8600537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:29.938 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.937+0000 7fa8867b1700 1 --2- 172.21.15.40:0/3359991739 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa860051320 0x7fa8600537e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fa87400b000 tx=0x7fa874006dc0).stop 2020-11-20T13:56:29.938 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.937+0000 7fa8867b1700 1 -- 172.21.15.40:0/3359991739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8801433b0 msgr2=0x7fa880131060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:29.938 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.937+0000 7fa8867b1700 1 --2- 172.21.15.40:0/3359991739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8801433b0 0x7fa880131060 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fa8780032b0 tx=0x7fa878016000).stop 2020-11-20T13:56:29.939 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.937+0000 7fa8867b1700 1 -- 172.21.15.40:0/3359991739 shutdown_connections 2020-11-20T13:56:29.939 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.937+0000 7fa8867b1700 1 --2- 172.21.15.40:0/3359991739 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa860051320 0x7fa8600537e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.939 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.937+0000 7fa8867b1700 1 --2- 172.21.15.40:0/3359991739 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa880067260 0x7fa880136d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.939 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.937+0000 7fa8867b1700 1 --2- 172.21.15.40:0/3359991739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8801433b0 0x7fa880131060 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.939 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.937+0000 7fa8867b1700 1 --2- 172.21.15.40:0/3359991739 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa880142a60 0x7fa880130b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.940 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.937+0000 7fa8867b1700 1 -- 172.21.15.40:0/3359991739 >> 172.21.15.40:0/3359991739 conn(0x7fa88005fe00 msgr2=0x7fa8800b66e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:29.940 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.937+0000 7fa8867b1700 1 -- 172.21.15.40:0/3359991739 shutdown_connections 2020-11-20T13:56:29.940 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:29.937+0000 7fa8867b1700 1 -- 172.21.15.40:0/3359991739 wait complete. 2020-11-20T13:56:29.949 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":4,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":22233},{"type":"v1","addr":"172.21.15.116:6801","nonce":22233}]},"active_addr":"172.21.15.116:6801/22233","active_change":"2020-11-20T13:56:08.285011+0000","active_mgr_features":4540138292840890367,"available":true,"standbys":[{"gid":4107,"name":"y","mgr_features":4540138292840890367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.6.2","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.6.2","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.116:0","nonce":3829889069}]},{"addrvec":[{"type":"v2","addr":"172.21.15.116:0","nonce":1044216152}]},{"addrvec":[{"type":"v2","addr":"172.21.15.116:0","nonce":47109010}]}]}} 2020-11-20T13:56:29.957 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2020-11-20T13:56:29.957 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2020-11-20T13:56:29.958 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-20T13:56:30.075 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.075+0000 7f02861f9700 1 Processor -- start 2020-11-20T13:56:30.076 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.075+0000 7f02861f9700 1 -- start start 2020-11-20T13:56:30.078 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.076+0000 7f02861f9700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f028009a280 0x7f02800b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.078 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.076+0000 7f02861f9700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f028009bc40 0x7f028009c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.078 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.076+0000 7f02861f9700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02800b6be0 con 0x7f02800b88c0 2020-11-20T13:56:30.078 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.076+0000 7f02861f9700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0280099000 con 0x7f028009a280 2020-11-20T13:56:30.078 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.076+0000 7f02861f9700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f028009c570 con 0x7f028009bc40 2020-11-20T13:56:30.079 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.076+0000 7f027ffff700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f028009bc40 0x7f028009c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.079 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.076+0000 7f027effd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f028009a280 0x7f02800b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.079 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.076+0000 7f027ffff700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f028009bc40 0x7f028009c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.40:39722/0 (socket says 172.21.15.40:39722) 2020-11-20T13:56:30.079 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.076+0000 7f027ffff700 1 -- 172.21.15.40:0/1084265620 learned_addr learned my addr 172.21.15.40:0/1084265620 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:30.079 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.077+0000 7f027effd700 1 -- 172.21.15.40:0/1084265620 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f028009bc40 msgr2=0x7f028009c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:30.080 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.077+0000 7f027effd700 1 --2- 172.21.15.40:0/1084265620 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f028009bc40 0x7f028009c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.080 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.077+0000 7f027effd700 1 -- 172.21.15.40:0/1084265620 >> v1:172.21.15.116:6789/0 conn(0x7f02800b88c0 legacy=0x7f028009b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.080 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.077+0000 7f027e7fc700 1 -- 172.21.15.40:0/1084265620 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 153547486 0 0) 0x7f02800b6be0 con 0x7f02800b88c0 2020-11-20T13:56:30.080 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.077+0000 7f027effd700 1 -- 172.21.15.40:0/1084265620 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02800b4250 con 0x7f028009a280 2020-11-20T13:56:30.080 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.077+0000 7f027ffff700 1 --2- 172.21.15.40:0/1084265620 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f028009bc40 0x7f028009c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:56:30.081 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.077+0000 7f027effd700 1 --2- 172.21.15.40:0/1084265620 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f028009a280 0x7f02800b8ca0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f0274002f30 tx=0x7f0274006dc0).ready entity=mon.1 client_cookie=5dc4cb784f02e11d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:30.081 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.077+0000 7f027e7fc700 1 -- 172.21.15.40:0/1084265620 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0274009760 con 0x7f028009a280 2020-11-20T13:56:30.081 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.077+0000 7f027e7fc700 1 -- 172.21.15.40:0/1084265620 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02740098c0 con 0x7f028009a280 2020-11-20T13:56:30.081 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.077+0000 7f027e7fc700 1 -- 172.21.15.40:0/1084265620 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0274009c20 con 0x7f028009a280 2020-11-20T13:56:30.082 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.077+0000 7f02861f9700 1 -- 172.21.15.40:0/1084265620 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f028009a280 msgr2=0x7f02800b8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.082 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.077+0000 7f02861f9700 1 --2- 172.21.15.40:0/1084265620 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f028009a280 0x7f02800b8ca0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f0274002f30 tx=0x7f0274006dc0).stop 2020-11-20T13:56:30.082 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.078+0000 7f02861f9700 1 -- 172.21.15.40:0/1084265620 shutdown_connections 2020-11-20T13:56:30.082 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.078+0000 7f02861f9700 1 --2- 172.21.15.40:0/1084265620 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f028009bc40 0x7f028009c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.083 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.078+0000 7f02861f9700 1 --2- 172.21.15.40:0/1084265620 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f028009a280 0x7f02800b8ca0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.083 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.078+0000 7f02861f9700 1 -- 172.21.15.40:0/1084265620 >> 172.21.15.40:0/1084265620 conn(0x7f02800ba380 msgr2=0x7f02800b6350 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:30.083 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.078+0000 7f02861f9700 1 -- 172.21.15.40:0/1084265620 shutdown_connections 2020-11-20T13:56:30.083 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.078+0000 7f02861f9700 1 -- 172.21.15.40:0/1084265620 wait complete. 2020-11-20T13:56:30.086 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.079+0000 7f02861f9700 1 Processor -- start 2020-11-20T13:56:30.086 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.079+0000 7f02861f9700 1 -- start start 2020-11-20T13:56:30.086 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.079+0000 7f02861f9700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f028009a280 0x7f028014a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.087 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.079+0000 7f02861f9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f028009bc40 0x7f02800b77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.087 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.080+0000 7f02861f9700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f02800b88c0 0x7f02800b7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.087 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.080+0000 7f02861f9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0280140680 con 0x7f028009bc40 2020-11-20T13:56:30.087 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.080+0000 7f02861f9700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0280156c40 con 0x7f028009a280 2020-11-20T13:56:30.087 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.080+0000 7f02861f9700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0280156ee0 con 0x7f02800b88c0 2020-11-20T13:56:30.088 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.080+0000 7f027effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f028009bc40 0x7f02800b77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.088 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.080+0000 7f027effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f028009bc40 0x7f02800b77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54652/0 (socket says 172.21.15.40:54652) 2020-11-20T13:56:30.088 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.080+0000 7f027effd700 1 -- 172.21.15.40:0/3412446530 learned_addr learned my addr 172.21.15.40:0/3412446530 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:30.088 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.080+0000 7f027f7fe700 1 --2- 172.21.15.40:0/3412446530 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f028009a280 0x7f028014a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.088 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.080+0000 7f027ffff700 1 --2- 172.21.15.40:0/3412446530 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f02800b88c0 0x7f02800b7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.089 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.080+0000 7f027effd700 1 -- 172.21.15.40:0/3412446530 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f02800b88c0 msgr2=0x7f02800b7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.089 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.080+0000 7f027effd700 1 --2- 172.21.15.40:0/3412446530 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f02800b88c0 0x7f02800b7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.089 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.080+0000 7f027effd700 1 -- 172.21.15.40:0/3412446530 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f028009a280 msgr2=0x7f028014a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.089 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.080+0000 7f027effd700 1 --2- 172.21.15.40:0/3412446530 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f028009a280 0x7f028014a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.089 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.080+0000 7f027effd700 1 -- 172.21.15.40:0/3412446530 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0280157180 con 0x7f028009bc40 2020-11-20T13:56:30.090 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.080+0000 7f027effd700 1 --2- 172.21.15.40:0/3412446530 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f028009bc40 0x7f02800b77b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f0274002f30 tx=0x7f0274015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:30.090 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.080+0000 7f027cff9700 1 -- 172.21.15.40:0/3412446530 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0274011950 con 0x7f028009bc40 2020-11-20T13:56:30.090 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.080+0000 7f027cff9700 1 -- 172.21.15.40:0/3412446530 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0274011ab0 con 0x7f028009bc40 2020-11-20T13:56:30.090 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.080+0000 7f02861f9700 1 -- 172.21.15.40:0/3412446530 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0280157620 con 0x7f028009bc40 2020-11-20T13:56:30.091 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.080+0000 7f027cff9700 1 -- 172.21.15.40:0/3412446530 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0274011e10 con 0x7f028009bc40 2020-11-20T13:56:30.091 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.080+0000 7f02861f9700 1 -- 172.21.15.40:0/3412446530 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0280156730 con 0x7f028009bc40 2020-11-20T13:56:30.091 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.081+0000 7f02861f9700 1 -- 172.21.15.40:0/3412446530 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0280057050 con 0x7f028009bc40 2020-11-20T13:56:30.092 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.081+0000 7f027cff9700 1 -- 172.21.15.40:0/3412446530 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f0274024030 con 0x7f028009bc40 2020-11-20T13:56:30.092 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.082+0000 7f027cff9700 1 --2- 172.21.15.40:0/3412446530 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f0260051320 0x7f02600537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.092 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.082+0000 7f027cff9700 1 -- 172.21.15.40:0/3412446530 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f0274069d40 con 0x7f028009bc40 2020-11-20T13:56:30.093 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.082+0000 7f027f7fe700 1 --2- 172.21.15.40:0/3412446530 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f0260051320 0x7f02600537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.093 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.082+0000 7f027f7fe700 1 --2- 172.21.15.40:0/3412446530 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f0260051320 0x7f02600537e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f027000d000 tx=0x7f02700078e0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:30.093 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.085+0000 7f027cff9700 1 -- 172.21.15.40:0/3412446530 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f0274041f60 con 0x7f028009bc40 2020-11-20T13:56:30.245 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.245+0000 7f02861f9700 1 -- 172.21.15.40:0/3412446530 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f02800b9690 con 0x7f028009bc40 2020-11-20T13:56:30.246 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.246+0000 7f027cff9700 1 -- 172.21.15.40:0/3412446530 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16347 (secure 0 0 0) 0x7f0274022020 con 0x7f028009bc40 2020-11-20T13:56:30.246 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:56:30.247 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":24,"fsid":"da1ed737-5174-430f-a3c9-56f6bbfad13c","created":"2020-11-20T13:56:04.990651+0000","modified":"2020-11-20T13:56:25.428032+0000","last_up_change":"2020-11-20T13:56:13.893026+0000","last_in_change":"2020-11-20T13:56:13.893026+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T13:56:08.330017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T13:56:17.939684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T13:56:20.563740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T13:56:23.062623+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9ea4c667-8938-46d6-a5ca-092d55c1f79e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":22825},{"type":"v1","addr":"172.21.15.40:6818","nonce":22825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":22825},{"type":"v1","addr":"172.21.15.40:6822","nonce":22825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":22825},{"type":"v1","addr":"172.21.15.40:6830","nonce":22825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":22825},{"type":"v1","addr":"172.21.15.40:6826","nonce":22825}]},"public_addr":"172.21.15.40:6818/22825","cluster_addr":"172.21.15.40:6822/22825","heartbeat_back_addr":"172.21.15.40:6830/22825","heartbeat_front_addr":"172.21.15.40:6826/22825","state":["exists","up"]},{"osd":1,"uuid":"515708d9-5087-4e55-8d14-cdea1c00eacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":22823},{"type":"v1","addr":"172.21.15.40:6801","nonce":22823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":22823},{"type":"v1","addr":"172.21.15.40:6803","nonce":22823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":22823},{"type":"v1","addr":"172.21.15.40:6807","nonce":22823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":22823},{"type":"v1","addr":"172.21.15.40:6805","nonce":22823}]},"public_addr":"172.21.15.40:6801/22823","cluster_addr":"172.21.15.40:6803/22823","heartbeat_back_addr":"172.21.15.40:6807/22823","heartbeat_front_addr":"172.21.15.40:6805/22823","state":["exists","up"]},{"osd":2,"uuid":"4d86bca7-743e-4983-b35f-36676c405b36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":22824},{"type":"v1","addr":"172.21.15.40:6809","nonce":22824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":22824},{"type":"v1","addr":"172.21.15.40:6811","nonce":22824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":22824},{"type":"v1","addr":"172.21.15.40:6815","nonce":22824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":22824},{"type":"v1","addr":"172.21.15.40:6813","nonce":22824}]},"public_addr":"172.21.15.40:6809/22824","cluster_addr":"172.21.15.40:6811/22824","heartbeat_back_addr":"172.21.15.40:6815/22824","heartbeat_front_addr":"172.21.15.40:6813/22824","state":["exists","up"]},{"osd":3,"uuid":"f4dc726b-6b34-4e8b-8eeb-87060b58002d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":22826},{"type":"v1","addr":"172.21.15.40:6819","nonce":22826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":22826},{"type":"v1","addr":"172.21.15.40:6823","nonce":22826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":22826},{"type":"v1","addr":"172.21.15.40:6831","nonce":22826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":22826},{"type":"v1","addr":"172.21.15.40:6827","nonce":22826}]},"public_addr":"172.21.15.40:6819/22826","cluster_addr":"172.21.15.40:6823/22826","heartbeat_back_addr":"172.21.15.40:6831/22826","heartbeat_front_addr":"172.21.15.40:6827/22826","state":["exists","up"]},{"osd":4,"uuid":"25a6d53b-d1f0-4498-b7d5-ffa38661d403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":23181},{"type":"v1","addr":"172.21.15.116:6819","nonce":23181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":23181},{"type":"v1","addr":"172.21.15.116:6821","nonce":23181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":23181},{"type":"v1","addr":"172.21.15.116:6825","nonce":23181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":23181},{"type":"v1","addr":"172.21.15.116:6823","nonce":23181}]},"public_addr":"172.21.15.116:6819/23181","cluster_addr":"172.21.15.116:6821/23181","heartbeat_back_addr":"172.21.15.116:6825/23181","heartbeat_front_addr":"172.21.15.116:6823/23181","state":["exists","up"]},{"osd":5,"uuid":"7d7a5af8-229b-4351-9a21-a3d509cd4635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":23171},{"type":"v1","addr":"172.21.15.116:6803","nonce":23171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":23171},{"type":"v1","addr":"172.21.15.116:6805","nonce":23171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":23171},{"type":"v1","addr":"172.21.15.116:6809","nonce":23171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":23171},{"type":"v1","addr":"172.21.15.116:6807","nonce":23171}]},"public_addr":"172.21.15.116:6803/23171","cluster_addr":"172.21.15.116:6805/23171","heartbeat_back_addr":"172.21.15.116:6809/23171","heartbeat_front_addr":"172.21.15.116:6807/23171","state":["exists","up"]},{"osd":6,"uuid":"8247d248-a140-4e36-9796-9847a97df58d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":23172},{"type":"v1","addr":"172.21.15.116:6811","nonce":23172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":23172},{"type":"v1","addr":"172.21.15.116:6813","nonce":23172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":23172},{"type":"v1","addr":"172.21.15.116:6817","nonce":23172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":23172},{"type":"v1","addr":"172.21.15.116:6815","nonce":23172}]},"public_addr":"172.21.15.116:6811/23172","cluster_addr":"172.21.15.116:6813/23172","heartbeat_back_addr":"172.21.15.116:6817/23172","heartbeat_front_addr":"172.21.15.116:6815/23172","state":["exists","up"]},{"osd":7,"uuid":"472803c4-70b8-499d-b18a-71aaeec6aab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":23196},{"type":"v1","addr":"172.21.15.116:6827","nonce":23196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":23196},{"type":"v1","addr":"172.21.15.116:6829","nonce":23196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":23196},{"type":"v1","addr":"172.21.15.116:6833","nonce":23196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":23196},{"type":"v1","addr":"172.21.15.116:6831","nonce":23196}]},"public_addr":"172.21.15.116:6827/23196","cluster_addr":"172.21.15.116:6829/23196","heartbeat_back_addr":"172.21.15.116:6833/23196","heartbeat_front_addr":"172.21.15.116:6831/23196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.029357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:11.989797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.091097+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.014766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.070860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.071864+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.063400+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.109028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-20T13:56:30.248 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.247+0000 7f02861f9700 1 -- 172.21.15.40:0/3412446530 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f0260051320 msgr2=0x7f02600537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.248 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.247+0000 7f02861f9700 1 --2- 172.21.15.40:0/3412446530 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f0260051320 0x7f02600537e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f027000d000 tx=0x7f02700078e0).stop 2020-11-20T13:56:30.249 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.247+0000 7f02861f9700 1 -- 172.21.15.40:0/3412446530 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f028009bc40 msgr2=0x7f02800b77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.249 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.247+0000 7f02861f9700 1 --2- 172.21.15.40:0/3412446530 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f028009bc40 0x7f02800b77b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f0274002f30 tx=0x7f0274015000).stop 2020-11-20T13:56:30.249 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.247+0000 7f02861f9700 1 -- 172.21.15.40:0/3412446530 shutdown_connections 2020-11-20T13:56:30.249 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.247+0000 7f02861f9700 1 --2- 172.21.15.40:0/3412446530 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f0260051320 0x7f02600537e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.250 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.247+0000 7f02861f9700 1 --2- 172.21.15.40:0/3412446530 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f02800b88c0 0x7f02800b7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.250 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.247+0000 7f02861f9700 1 --2- 172.21.15.40:0/3412446530 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f028009a280 0x7f028014a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.250 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.247+0000 7f02861f9700 1 --2- 172.21.15.40:0/3412446530 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f028009bc40 0x7f02800b77b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.250 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.247+0000 7f02861f9700 1 -- 172.21.15.40:0/3412446530 >> 172.21.15.40:0/3412446530 conn(0x7f02800ba380 msgr2=0x7f02800b6350 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:30.250 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.247+0000 7f02861f9700 1 -- 172.21.15.40:0/3412446530 shutdown_connections 2020-11-20T13:56:30.251 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.247+0000 7f02861f9700 1 -- 172.21.15.40:0/3412446530 wait complete. 2020-11-20T13:56:30.262 INFO:tasks.ceph.ceph_manager.ceph:all up! 2020-11-20T13:56:30.262 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-20T13:56:30.381 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.376+0000 7fc0bcc3e700 1 Processor -- start 2020-11-20T13:56:30.381 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.376+0000 7fc0bcc3e700 1 -- start start 2020-11-20T13:56:30.382 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.377+0000 7fc0bcc3e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc0b80665b0 0x7fc0b8146450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.382 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.377+0000 7fc0bcc3e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0b81457d0 con 0x7fc0b80b8800 2020-11-20T13:56:30.382 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.377+0000 7fc0bcc3e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0b8145a70 con 0x7fc0b812b9d0 2020-11-20T13:56:30.382 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.377+0000 7fc0bcc3e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0b8145d10 con 0x7fc0b80665b0 2020-11-20T13:56:30.382 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.377+0000 7fc0b659c700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc0b80b8800 0x7fc0b812b2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45254/0 (socket says 172.21.15.40:45254) 2020-11-20T13:56:30.383 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.377+0000 7fc0b659c700 1 -- 172.21.15.40:0/821443523 learned_addr learned my addr 172.21.15.40:0/821443523 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:30.383 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.378+0000 7fc0b559a700 1 -- 172.21.15.40:0/821443523 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2997720241 0 0) 0x7fc0b81457d0 con 0x7fc0b80b8800 2020-11-20T13:56:30.383 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.378+0000 7fc0b5d9b700 1 --2- 172.21.15.40:0/821443523 >> v2:172.21.15.116:3300/0 conn(0x7fc0b80665b0 0x7fc0b8146450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.383 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.378+0000 7fc0b559a700 1 -- 172.21.15.40:0/821443523 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc098003230 con 0x7fc0b80b8800 2020-11-20T13:56:30.383 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.378+0000 7fc0b559a700 1 -- 172.21.15.40:0/821443523 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 121893816 0 0) 0x7fc098003230 con 0x7fc0b80b8800 2020-11-20T13:56:30.384 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.378+0000 7fc0b559a700 1 -- 172.21.15.40:0/821443523 >> v2:172.21.15.116:3300/0 conn(0x7fc0b80665b0 msgr2=0x7fc0b8146450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:30.384 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.378+0000 7fc0b559a700 1 --2- 172.21.15.40:0/821443523 >> v2:172.21.15.116:3300/0 conn(0x7fc0b80665b0 0x7fc0b8146450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.384 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.378+0000 7fc0b559a700 1 -- 172.21.15.40:0/821443523 >> v1:172.21.15.116:6789/0 conn(0x7fc0b812b9d0 legacy=0x7fc0b8064ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.384 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.378+0000 7fc0b559a700 1 -- 172.21.15.40:0/821443523 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b8145fb0 con 0x7fc0b80b8800 2020-11-20T13:56:30.384 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.378+0000 7fc0b559a700 1 -- 172.21.15.40:0/821443523 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2366654341 0 0) 0x7fc0a0001850 con 0x7fc0b80b8800 2020-11-20T13:56:30.385 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.378+0000 7fc0b559a700 1 -- 172.21.15.40:0/821443523 >> v1:172.21.15.40:6789/0 conn(0x7fc0b80b8800 legacy=0x7fc0b812b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.385 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.378+0000 7fc0b559a700 1 --2- 172.21.15.40:0/821443523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc098003720 0x7fc098003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.387 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.378+0000 7fc0b559a700 1 --2- 172.21.15.40:0/821443523 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0980044d0 0x7fc098004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.387 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.378+0000 7fc0b559a700 1 --2- 172.21.15.40:0/821443523 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc098004e40 0x7fc09800b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.387 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.378+0000 7fc0b559a700 1 -- 172.21.15.40:0/821443523 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b8145d10 con 0x7fc098003720 2020-11-20T13:56:30.387 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.378+0000 7fc0b559a700 1 -- 172.21.15.40:0/821443523 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc098004340 con 0x7fc0980044d0 2020-11-20T13:56:30.388 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.378+0000 7fc0b559a700 1 -- 172.21.15.40:0/821443523 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc09800bc40 con 0x7fc098004e40 2020-11-20T13:56:30.388 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.378+0000 7fc0b659c700 1 --2- 172.21.15.40:0/821443523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc098003720 0x7fc098003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.388 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.378+0000 7fc0b5d9b700 1 --2- 172.21.15.40:0/821443523 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0980044d0 0x7fc098004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.388 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.378+0000 7fc0b6d9d700 1 --2- 172.21.15.40:0/821443523 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc098004e40 0x7fc09800b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.389 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.378+0000 7fc0b659c700 1 -- 172.21.15.40:0/821443523 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc098004e40 msgr2=0x7fc09800b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:30.389 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.378+0000 7fc0b659c700 1 --2- 172.21.15.40:0/821443523 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc098004e40 0x7fc09800b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.389 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.378+0000 7fc0b659c700 1 -- 172.21.15.40:0/821443523 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0980044d0 msgr2=0x7fc098004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:30.389 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.378+0000 7fc0b659c700 1 --2- 172.21.15.40:0/821443523 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0980044d0 0x7fc098004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.389 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.378+0000 7fc0b659c700 1 -- 172.21.15.40:0/821443523 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc098003230 con 0x7fc098003720 2020-11-20T13:56:30.390 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.378+0000 7fc0b6d9d700 1 --2- 172.21.15.40:0/821443523 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc098004e40 0x7fc09800b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:30.391 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.378+0000 7fc0b6d9d700 1 -- 172.21.15.40:0/821443523 reap_dead start 2020-11-20T13:56:30.391 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.379+0000 7fc0b659c700 1 --2- 172.21.15.40:0/821443523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc098003720 0x7fc098003b40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fc0a0001560 tx=0x7fc0b8145960).ready entity=mon.0 client_cookie=278252031ba3e9b1 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:30.393 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.379+0000 7fc0b559a700 1 -- 172.21.15.40:0/821443523 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc0a00070d0 con 0x7fc098003720 2020-11-20T13:56:30.393 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.379+0000 7fc0b559a700 1 -- 172.21.15.40:0/821443523 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0a0007230 con 0x7fc098003720 2020-11-20T13:56:30.393 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.379+0000 7fc0b559a700 1 -- 172.21.15.40:0/821443523 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc0a00056a0 con 0x7fc098003720 2020-11-20T13:56:30.393 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.379+0000 7fc0bcc3e700 1 -- 172.21.15.40:0/821443523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc098003720 msgr2=0x7fc098003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.394 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.379+0000 7fc0bcc3e700 1 --2- 172.21.15.40:0/821443523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc098003720 0x7fc098003b40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fc0a0001560 tx=0x7fc0b8145960).stop 2020-11-20T13:56:30.394 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.379+0000 7fc0bcc3e700 1 -- 172.21.15.40:0/821443523 shutdown_connections 2020-11-20T13:56:30.394 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.379+0000 7fc0bcc3e700 1 --2- 172.21.15.40:0/821443523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc098003720 0x7fc098003b40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.395 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.379+0000 7fc0bcc3e700 1 -- 172.21.15.40:0/821443523 >> 172.21.15.40:0/821443523 conn(0x7fc0b805c190 msgr2=0x7fc0b805c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:30.395 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.379+0000 7fc0bcc3e700 1 -- 172.21.15.40:0/821443523 shutdown_connections 2020-11-20T13:56:30.395 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.379+0000 7fc0bcc3e700 1 -- 172.21.15.40:0/821443523 wait complete. 2020-11-20T13:56:30.396 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.380+0000 7fc0bcc3e700 1 Processor -- start 2020-11-20T13:56:30.396 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.380+0000 7fc0bcc3e700 1 -- start start 2020-11-20T13:56:30.407 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.381+0000 7fc0bcc3e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc098003720 0x7fc0b812a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.407 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.381+0000 7fc0bcc3e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0b80b43b0 0x7fc0b812a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.408 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.381+0000 7fc0bcc3e700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc0b8157d00 0x7fc0b81581b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.408 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.381+0000 7fc0bcc3e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b80b39d0 con 0x7fc0b80b43b0 2020-11-20T13:56:30.408 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.381+0000 7fc0bcc3e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b80b3c70 con 0x7fc098003720 2020-11-20T13:56:30.409 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.381+0000 7fc0bcc3e700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b80b3f10 con 0x7fc0b8157d00 2020-11-20T13:56:30.409 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.381+0000 7fc0b5d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0b80b43b0 0x7fc0b812a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.409 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.381+0000 7fc0b5d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0b80b43b0 0x7fc0b812a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54672/0 (socket says 172.21.15.40:54672) 2020-11-20T13:56:30.409 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.381+0000 7fc0b5d9b700 1 -- 172.21.15.40:0/2899591572 learned_addr learned my addr 172.21.15.40:0/2899591572 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:30.409 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.381+0000 7fc0b659c700 1 --2- 172.21.15.40:0/2899591572 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc098003720 0x7fc0b812a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.412 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.381+0000 7fc0b6d9d700 1 --2- 172.21.15.40:0/2899591572 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc0b8157d00 0x7fc0b81581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.413 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.381+0000 7fc0b5d9b700 1 -- 172.21.15.40:0/2899591572 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc0b8157d00 msgr2=0x7fc0b81581b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.413 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.381+0000 7fc0b5d9b700 1 --2- 172.21.15.40:0/2899591572 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc0b8157d00 0x7fc0b81581b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.413 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.381+0000 7fc0b5d9b700 1 -- 172.21.15.40:0/2899591572 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc098003720 msgr2=0x7fc0b812a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.413 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.381+0000 7fc0b5d9b700 1 --2- 172.21.15.40:0/2899591572 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc098003720 0x7fc0b812a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.413 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.381+0000 7fc0b5d9b700 1 -- 172.21.15.40:0/2899591572 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b80b5a50 con 0x7fc0b80b43b0 2020-11-20T13:56:30.414 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.381+0000 7fc0b5d9b700 1 --2- 172.21.15.40:0/2899591572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0b80b43b0 0x7fc0b812a9f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fc0a800b000 tx=0x7fc0a80091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:30.414 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.382+0000 7fc0af7fe700 1 -- 172.21.15.40:0/2899591572 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc0a800fab0 con 0x7fc0b80b43b0 2020-11-20T13:56:30.414 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.382+0000 7fc0af7fe700 1 -- 172.21.15.40:0/2899591572 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0a800fc10 con 0x7fc0b80b43b0 2020-11-20T13:56:30.414 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.382+0000 7fc0bcc3e700 1 -- 172.21.15.40:0/2899591572 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0b81586a0 con 0x7fc0b80b43b0 2020-11-20T13:56:30.415 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.382+0000 7fc0af7fe700 1 -- 172.21.15.40:0/2899591572 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc0a8014440 con 0x7fc0b80b43b0 2020-11-20T13:56:30.415 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.382+0000 7fc0bcc3e700 1 -- 172.21.15.40:0/2899591572 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0b812bff0 con 0x7fc0b80b43b0 2020-11-20T13:56:30.415 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.383+0000 7fc0bcc3e700 1 -- 172.21.15.40:0/2899591572 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0b80b40d0 con 0x7fc0b80b43b0 2020-11-20T13:56:30.415 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.383+0000 7fc0af7fe700 1 -- 172.21.15.40:0/2899591572 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fc0a8024070 con 0x7fc0b80b43b0 2020-11-20T13:56:30.418 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.383+0000 7fc0af7fe700 1 --2- 172.21.15.40:0/2899591572 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fc09c051320 0x7fc09c0537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.418 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.383+0000 7fc0af7fe700 1 -- 172.21.15.40:0/2899591572 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fc0a8069a10 con 0x7fc0b80b43b0 2020-11-20T13:56:30.418 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.383+0000 7fc0b659c700 1 --2- 172.21.15.40:0/2899591572 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fc09c051320 0x7fc09c0537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.419 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.383+0000 7fc0b659c700 1 --2- 172.21.15.40:0/2899591572 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fc09c051320 0x7fc09c0537e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fc0a001b000 tx=0x7fc0a0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:30.420 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.386+0000 7fc0af7fe700 1 -- 172.21.15.40:0/2899591572 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fc0a8041c30 con 0x7fc0b80b43b0 2020-11-20T13:56:30.548 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.548+0000 7fc0bcc3e700 1 -- 172.21.15.40:0/2899591572 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc0b8057050 con 0x7fc0b80b43b0 2020-11-20T13:56:30.549 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.548+0000 7fc0af7fe700 1 -- 172.21.15.40:0/2899591572 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16347 (secure 0 0 0) 0x7fc0a8045250 con 0x7fc0b80b43b0 2020-11-20T13:56:30.549 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:56:30.549 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":24,"fsid":"da1ed737-5174-430f-a3c9-56f6bbfad13c","created":"2020-11-20T13:56:04.990651+0000","modified":"2020-11-20T13:56:25.428032+0000","last_up_change":"2020-11-20T13:56:13.893026+0000","last_in_change":"2020-11-20T13:56:13.893026+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T13:56:08.330017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T13:56:17.939684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T13:56:20.563740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T13:56:23.062623+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9ea4c667-8938-46d6-a5ca-092d55c1f79e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":22825},{"type":"v1","addr":"172.21.15.40:6818","nonce":22825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":22825},{"type":"v1","addr":"172.21.15.40:6822","nonce":22825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":22825},{"type":"v1","addr":"172.21.15.40:6830","nonce":22825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":22825},{"type":"v1","addr":"172.21.15.40:6826","nonce":22825}]},"public_addr":"172.21.15.40:6818/22825","cluster_addr":"172.21.15.40:6822/22825","heartbeat_back_addr":"172.21.15.40:6830/22825","heartbeat_front_addr":"172.21.15.40:6826/22825","state":["exists","up"]},{"osd":1,"uuid":"515708d9-5087-4e55-8d14-cdea1c00eacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":22823},{"type":"v1","addr":"172.21.15.40:6801","nonce":22823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":22823},{"type":"v1","addr":"172.21.15.40:6803","nonce":22823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":22823},{"type":"v1","addr":"172.21.15.40:6807","nonce":22823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":22823},{"type":"v1","addr":"172.21.15.40:6805","nonce":22823}]},"public_addr":"172.21.15.40:6801/22823","cluster_addr":"172.21.15.40:6803/22823","heartbeat_back_addr":"172.21.15.40:6807/22823","heartbeat_front_addr":"172.21.15.40:6805/22823","state":["exists","up"]},{"osd":2,"uuid":"4d86bca7-743e-4983-b35f-36676c405b36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":22824},{"type":"v1","addr":"172.21.15.40:6809","nonce":22824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":22824},{"type":"v1","addr":"172.21.15.40:6811","nonce":22824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":22824},{"type":"v1","addr":"172.21.15.40:6815","nonce":22824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":22824},{"type":"v1","addr":"172.21.15.40:6813","nonce":22824}]},"public_addr":"172.21.15.40:6809/22824","cluster_addr":"172.21.15.40:6811/22824","heartbeat_back_addr":"172.21.15.40:6815/22824","heartbeat_front_addr":"172.21.15.40:6813/22824","state":["exists","up"]},{"osd":3,"uuid":"f4dc726b-6b34-4e8b-8eeb-87060b58002d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":22826},{"type":"v1","addr":"172.21.15.40:6819","nonce":22826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":22826},{"type":"v1","addr":"172.21.15.40:6823","nonce":22826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":22826},{"type":"v1","addr":"172.21.15.40:6831","nonce":22826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":22826},{"type":"v1","addr":"172.21.15.40:6827","nonce":22826}]},"public_addr":"172.21.15.40:6819/22826","cluster_addr":"172.21.15.40:6823/22826","heartbeat_back_addr":"172.21.15.40:6831/22826","heartbeat_front_addr":"172.21.15.40:6827/22826","state":["exists","up"]},{"osd":4,"uuid":"25a6d53b-d1f0-4498-b7d5-ffa38661d403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":23181},{"type":"v1","addr":"172.21.15.116:6819","nonce":23181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":23181},{"type":"v1","addr":"172.21.15.116:6821","nonce":23181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":23181},{"type":"v1","addr":"172.21.15.116:6825","nonce":23181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":23181},{"type":"v1","addr":"172.21.15.116:6823","nonce":23181}]},"public_addr":"172.21.15.116:6819/23181","cluster_addr":"172.21.15.116:6821/23181","heartbeat_back_addr":"172.21.15.116:6825/23181","heartbeat_front_addr":"172.21.15.116:6823/23181","state":["exists","up"]},{"osd":5,"uuid":"7d7a5af8-229b-4351-9a21-a3d509cd4635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":23171},{"type":"v1","addr":"172.21.15.116:6803","nonce":23171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":23171},{"type":"v1","addr":"172.21.15.116:6805","nonce":23171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":23171},{"type":"v1","addr":"172.21.15.116:6809","nonce":23171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":23171},{"type":"v1","addr":"172.21.15.116:6807","nonce":23171}]},"public_addr":"172.21.15.116:6803/23171","cluster_addr":"172.21.15.116:6805/23171","heartbeat_back_addr":"172.21.15.116:6809/23171","heartbeat_front_addr":"172.21.15.116:6807/23171","state":["exists","up"]},{"osd":6,"uuid":"8247d248-a140-4e36-9796-9847a97df58d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":23172},{"type":"v1","addr":"172.21.15.116:6811","nonce":23172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":23172},{"type":"v1","addr":"172.21.15.116:6813","nonce":23172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":23172},{"type":"v1","addr":"172.21.15.116:6817","nonce":23172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":23172},{"type":"v1","addr":"172.21.15.116:6815","nonce":23172}]},"public_addr":"172.21.15.116:6811/23172","cluster_addr":"172.21.15.116:6813/23172","heartbeat_back_addr":"172.21.15.116:6817/23172","heartbeat_front_addr":"172.21.15.116:6815/23172","state":["exists","up"]},{"osd":7,"uuid":"472803c4-70b8-499d-b18a-71aaeec6aab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":23196},{"type":"v1","addr":"172.21.15.116:6827","nonce":23196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":23196},{"type":"v1","addr":"172.21.15.116:6829","nonce":23196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":23196},{"type":"v1","addr":"172.21.15.116:6833","nonce":23196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":23196},{"type":"v1","addr":"172.21.15.116:6831","nonce":23196}]},"public_addr":"172.21.15.116:6827/23196","cluster_addr":"172.21.15.116:6829/23196","heartbeat_back_addr":"172.21.15.116:6833/23196","heartbeat_front_addr":"172.21.15.116:6831/23196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.029357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:11.989797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.091097+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.014766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.070860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.071864+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.063400+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.109028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-20T13:56:30.550 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.550+0000 7fc0bcc3e700 1 -- 172.21.15.40:0/2899591572 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fc09c051320 msgr2=0x7fc09c0537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.551 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.550+0000 7fc0bcc3e700 1 --2- 172.21.15.40:0/2899591572 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fc09c051320 0x7fc09c0537e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fc0a001b000 tx=0x7fc0a0015000).stop 2020-11-20T13:56:30.551 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.550+0000 7fc0bcc3e700 1 -- 172.21.15.40:0/2899591572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0b80b43b0 msgr2=0x7fc0b812a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.551 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.550+0000 7fc0bcc3e700 1 --2- 172.21.15.40:0/2899591572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0b80b43b0 0x7fc0b812a9f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fc0a800b000 tx=0x7fc0a80091a0).stop 2020-11-20T13:56:30.551 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.550+0000 7fc0bcc3e700 1 -- 172.21.15.40:0/2899591572 shutdown_connections 2020-11-20T13:56:30.551 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.550+0000 7fc0bcc3e700 1 --2- 172.21.15.40:0/2899591572 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fc09c051320 0x7fc09c0537e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.552 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.550+0000 7fc0bcc3e700 1 --2- 172.21.15.40:0/2899591572 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc0b8157d00 0x7fc0b81581b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.552 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.550+0000 7fc0bcc3e700 1 --2- 172.21.15.40:0/2899591572 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc098003720 0x7fc0b812a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.552 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.550+0000 7fc0bcc3e700 1 --2- 172.21.15.40:0/2899591572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0b80b43b0 0x7fc0b812a9f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.553 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.550+0000 7fc0bcc3e700 1 -- 172.21.15.40:0/2899591572 >> 172.21.15.40:0/2899591572 conn(0x7fc0b805c190 msgr2=0x7fc0b8145330 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:30.553 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.550+0000 7fc0bcc3e700 1 -- 172.21.15.40:0/2899591572 shutdown_connections 2020-11-20T13:56:30.553 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.550+0000 7fc0bcc3e700 1 -- 172.21.15.40:0/2899591572 wait complete. 2020-11-20T13:56:30.561 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.0 flush_pg_stats 2020-11-20T13:56:30.677 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.677+0000 7fb501c53700 1 Processor -- start 2020-11-20T13:56:30.677 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.677+0000 7fb501c53700 1 -- start start 2020-11-20T13:56:30.679 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.677+0000 7fb501c53700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb4fc0b6c80 0x7fb4fc0b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.680 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.677+0000 7fb501c53700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb4fc144bd0 0x7fb4fc0b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.680 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.677+0000 7fb501c53700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4fc0b0d10 con 0x7fb4fc0b45e0 2020-11-20T13:56:30.680 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.677+0000 7fb501c53700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4fc145030 con 0x7fb4fc144bd0 2020-11-20T13:56:30.680 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.677+0000 7fb501c53700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc0b0fb0 con 0x7fb4fc0b6c80 2020-11-20T13:56:30.680 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.678+0000 7fb4fbfff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb4fc0b45e0 0x7fb4fc0b1450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45274/0 (socket says 172.21.15.40:45274) 2020-11-20T13:56:30.681 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.678+0000 7fb4fbfff700 1 -- 172.21.15.40:0/1276010424 learned_addr learned my addr 172.21.15.40:0/1276010424 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:30.681 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.678+0000 7fb4fb7fe700 1 --2- 172.21.15.40:0/1276010424 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb4fc0b6c80 0x7fb4fc0b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.681 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.678+0000 7fb4f3fff700 1 --2- 172.21.15.40:0/1276010424 >> v2:172.21.15.116:3300/0 conn(0x7fb4fc144bd0 0x7fb4fc0b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.681 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.678+0000 7fb4faffd700 1 -- 172.21.15.40:0/1276010424 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3181737684 0 0) 0x7fb4fc0b0d10 con 0x7fb4fc0b45e0 2020-11-20T13:56:30.681 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.678+0000 7fb4faffd700 1 -- 172.21.15.40:0/1276010424 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4dc003230 con 0x7fb4fc0b45e0 2020-11-20T13:56:30.682 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.678+0000 7fb4fb7fe700 1 -- 172.21.15.40:0/1276010424 >> v2:172.21.15.116:3300/0 conn(0x7fb4fc144bd0 msgr2=0x7fb4fc0b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:30.682 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.678+0000 7fb4fb7fe700 1 --2- 172.21.15.40:0/1276010424 >> v2:172.21.15.116:3300/0 conn(0x7fb4fc144bd0 0x7fb4fc0b40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.682 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.678+0000 7fb4fb7fe700 1 -- 172.21.15.40:0/1276010424 >> v1:172.21.15.40:6789/0 conn(0x7fb4fc0b45e0 legacy=0x7fb4fc0b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.682 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.678+0000 7fb4fb7fe700 1 -- 172.21.15.40:0/1276010424 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4fc0b7060 con 0x7fb4fc0b6c80 2020-11-20T13:56:30.683 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.678+0000 7fb4f3fff700 1 --2- 172.21.15.40:0/1276010424 >> v2:172.21.15.116:3300/0 conn(0x7fb4fc144bd0 0x7fb4fc0b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:56:30.683 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.678+0000 7fb4fb7fe700 1 --2- 172.21.15.40:0/1276010424 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb4fc0b6c80 0x7fb4fc0b31f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fb4ec002f30 tx=0x7fb4ec006d60).ready entity=mon.2 client_cookie=d5403f0284d3e974 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:30.683 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.678+0000 7fb4faffd700 1 -- 172.21.15.40:0/1276010424 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb4ec009580 con 0x7fb4fc0b6c80 2020-11-20T13:56:30.683 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.678+0000 7fb4faffd700 1 -- 172.21.15.40:0/1276010424 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ec0096e0 con 0x7fb4fc0b6c80 2020-11-20T13:56:30.684 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.678+0000 7fb4faffd700 1 -- 172.21.15.40:0/1276010424 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb4ec009850 con 0x7fb4fc0b6c80 2020-11-20T13:56:30.684 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.678+0000 7fb501c53700 1 -- 172.21.15.40:0/1276010424 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb4fc0b6c80 msgr2=0x7fb4fc0b31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.685 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.678+0000 7fb501c53700 1 --2- 172.21.15.40:0/1276010424 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb4fc0b6c80 0x7fb4fc0b31f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fb4ec002f30 tx=0x7fb4ec006d60).stop 2020-11-20T13:56:30.685 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.679+0000 7fb501c53700 1 -- 172.21.15.40:0/1276010424 shutdown_connections 2020-11-20T13:56:30.685 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.679+0000 7fb501c53700 1 --2- 172.21.15.40:0/1276010424 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb4fc0b6c80 0x7fb4fc0b31f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.686 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.679+0000 7fb501c53700 1 --2- 172.21.15.40:0/1276010424 >> v2:172.21.15.116:3300/0 conn(0x7fb4fc144bd0 0x7fb4fc0b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.687 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.679+0000 7fb501c53700 1 -- 172.21.15.40:0/1276010424 >> 172.21.15.40:0/1276010424 conn(0x7fb4fc0ba1d0 msgr2=0x7fb4fc0b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:30.687 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.679+0000 7fb501c53700 1 -- 172.21.15.40:0/1276010424 shutdown_connections 2020-11-20T13:56:30.687 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.679+0000 7fb501c53700 1 -- 172.21.15.40:0/1276010424 wait complete. 2020-11-20T13:56:30.690 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.680+0000 7fb501c53700 1 Processor -- start 2020-11-20T13:56:30.690 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.680+0000 7fb501c53700 1 -- start start 2020-11-20T13:56:30.691 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.681+0000 7fb501c53700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4fc0b45e0 0x7fb4fc0669a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.691 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.681+0000 7fb501c53700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb4fc0b6c80 0x7fb4fc067040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.691 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.681+0000 7fb501c53700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb4fc144bd0 0x7fb4fc0657d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.691 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.681+0000 7fb501c53700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc065400 con 0x7fb4fc0b45e0 2020-11-20T13:56:30.692 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.681+0000 7fb501c53700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc0663b0 con 0x7fb4fc144bd0 2020-11-20T13:56:30.692 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.681+0000 7fb501c53700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc064f60 con 0x7fb4fc0b6c80 2020-11-20T13:56:30.692 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.681+0000 7fb4fb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4fc0b45e0 0x7fb4fc0669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.692 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.681+0000 7fb4fb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4fc0b45e0 0x7fb4fc0669a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54684/0 (socket says 172.21.15.40:54684) 2020-11-20T13:56:30.692 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.681+0000 7fb4fb7fe700 1 -- 172.21.15.40:0/3528683078 learned_addr learned my addr 172.21.15.40:0/3528683078 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:30.693 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.681+0000 7fb4f3fff700 1 --2- 172.21.15.40:0/3528683078 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb4fc0b6c80 0x7fb4fc067040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.693 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.681+0000 7fb4fbfff700 1 --2- 172.21.15.40:0/3528683078 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb4fc144bd0 0x7fb4fc0657d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.693 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.681+0000 7fb4fb7fe700 1 -- 172.21.15.40:0/3528683078 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb4fc0b6c80 msgr2=0x7fb4fc067040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.693 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.681+0000 7fb4fb7fe700 1 --2- 172.21.15.40:0/3528683078 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb4fc0b6c80 0x7fb4fc067040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.693 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.681+0000 7fb4fb7fe700 1 -- 172.21.15.40:0/3528683078 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb4fc144bd0 msgr2=0x7fb4fc0657d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.694 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.681+0000 7fb4fb7fe700 1 --2- 172.21.15.40:0/3528683078 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb4fc144bd0 0x7fb4fc0657d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.694 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.681+0000 7fb4fb7fe700 1 -- 172.21.15.40:0/3528683078 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4fc067530 con 0x7fb4fc0b45e0 2020-11-20T13:56:30.694 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.681+0000 7fb4f3fff700 1 --2- 172.21.15.40:0/3528683078 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb4fc0b6c80 0x7fb4fc067040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:30.694 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.681+0000 7fb4fb7fe700 1 --2- 172.21.15.40:0/3528683078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4fc0b45e0 0x7fb4fc0669a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fb4ec002f30 tx=0x7fb4ec016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:30.694 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.681+0000 7fb4f97fa700 1 -- 172.21.15.40:0/3528683078 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb4ec011ce0 con 0x7fb4fc0b45e0 2020-11-20T13:56:30.695 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.681+0000 7fb4f97fa700 1 -- 172.21.15.40:0/3528683078 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ec011e40 con 0x7fb4fc0b45e0 2020-11-20T13:56:30.695 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.681+0000 7fb501c53700 1 -- 172.21.15.40:0/3528683078 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4fc1437d0 con 0x7fb4fc0b45e0 2020-11-20T13:56:30.695 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.681+0000 7fb4f97fa700 1 -- 172.21.15.40:0/3528683078 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb4ec021670 con 0x7fb4fc0b45e0 2020-11-20T13:56:30.695 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.681+0000 7fb501c53700 1 -- 172.21.15.40:0/3528683078 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4fc065e80 con 0x7fb4fc0b45e0 2020-11-20T13:56:30.696 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.682+0000 7fb501c53700 1 -- 172.21.15.40:0/3528683078 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb4fc065120 con 0x7fb4fc0b45e0 2020-11-20T13:56:30.696 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.683+0000 7fb4f97fa700 1 -- 172.21.15.40:0/3528683078 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fb4ec00f070 con 0x7fb4fc0b45e0 2020-11-20T13:56:30.696 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.683+0000 7fb4f97fa700 1 --2- 172.21.15.40:0/3528683078 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fb4e0051320 0x7fb4e00537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.696 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.683+0000 7fb4f97fa700 1 -- 172.21.15.40:0/3528683078 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fb4ec069d20 con 0x7fb4fc0b45e0 2020-11-20T13:56:30.696 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.683+0000 7fb4f97fa700 1 --2- 172.21.15.40:0/3528683078 >> [v2:172.21.15.40:6816/22825,v1:172.21.15.40:6818/22825] conn(0x7fb4e0058e30 0x7fb4e005b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.697 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.683+0000 7fb4f97fa700 1 -- 172.21.15.40:0/3528683078 --> [v2:172.21.15.40:6816/22825,v1:172.21.15.40:6818/22825] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb4e005b8a0 con 0x7fb4e0058e30 2020-11-20T13:56:30.697 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.683+0000 7fb4f97fa700 1 -- 172.21.15.40:0/3528683078 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4ec06c3b0 con 0x7fb4fc0b45e0 2020-11-20T13:56:30.697 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.683+0000 7fb4f3fff700 1 --2- 172.21.15.40:0/3528683078 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fb4e0051320 0x7fb4e00537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.697 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.683+0000 7fb4fbfff700 1 --2- 172.21.15.40:0/3528683078 >> [v2:172.21.15.40:6816/22825,v1:172.21.15.40:6818/22825] conn(0x7fb4e0058e30 0x7fb4e005b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.697 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.683+0000 7fb4f3fff700 1 --2- 172.21.15.40:0/3528683078 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fb4e0051320 0x7fb4e00537e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fb4e400b000 tx=0x7fb4e4008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:30.698 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.683+0000 7fb4fbfff700 1 --2- 172.21.15.40:0/3528683078 >> [v2:172.21.15.40:6816/22825,v1:172.21.15.40:6818/22825] conn(0x7fb4e0058e30 0x7fb4e005b250 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:30.698 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.684+0000 7fb4f97fa700 1 -- 172.21.15.40:0/3528683078 <== osd.0 v2:172.21.15.40:6816/22825 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7fb4e005b8a0 con 0x7fb4e0058e30 2020-11-20T13:56:30.704 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.702+0000 7fb501c53700 1 -- 172.21.15.40:0/3528683078 --> [v2:172.21.15.40:6816/22825,v1:172.21.15.40:6818/22825] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb4fc0aeff0 con 0x7fb4e0058e30 2020-11-20T13:56:30.704 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.702+0000 7fb4f97fa700 1 -- 172.21.15.40:0/3528683078 <== osd.0 v2:172.21.15.40:6816/22825 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb4fc0aeff0 con 0x7fb4e0058e30 2020-11-20T13:56:30.705 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.702+0000 7fb501c53700 1 -- 172.21.15.40:0/3528683078 >> [v2:172.21.15.40:6816/22825,v1:172.21.15.40:6818/22825] conn(0x7fb4e0058e30 msgr2=0x7fb4e005b250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.705 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.702+0000 7fb501c53700 1 --2- 172.21.15.40:0/3528683078 >> [v2:172.21.15.40:6816/22825,v1:172.21.15.40:6818/22825] conn(0x7fb4e0058e30 0x7fb4e005b250 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.705 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.703+0000 7fb501c53700 1 -- 172.21.15.40:0/3528683078 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fb4e0051320 msgr2=0x7fb4e00537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.705 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.703+0000 7fb501c53700 1 --2- 172.21.15.40:0/3528683078 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fb4e0051320 0x7fb4e00537e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fb4e400b000 tx=0x7fb4e4008040).stop 2020-11-20T13:56:30.707 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.703+0000 7fb501c53700 1 -- 172.21.15.40:0/3528683078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4fc0b45e0 msgr2=0x7fb4fc0669a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.707 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.703+0000 7fb501c53700 1 --2- 172.21.15.40:0/3528683078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4fc0b45e0 0x7fb4fc0669a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fb4ec002f30 tx=0x7fb4ec016000).stop 2020-11-20T13:56:30.707 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.703+0000 7fb4fbfff700 1 -- 172.21.15.40:0/3528683078 reap_dead start 2020-11-20T13:56:30.707 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.703+0000 7fb501c53700 1 -- 172.21.15.40:0/3528683078 shutdown_connections 2020-11-20T13:56:30.708 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.703+0000 7fb501c53700 1 -- 172.21.15.40:0/3528683078 >> 172.21.15.40:0/3528683078 conn(0x7fb4fc0ba1d0 msgr2=0x7fb4fc0b4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:30.708 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.703+0000 7fb501c53700 1 -- 172.21.15.40:0/3528683078 shutdown_connections 2020-11-20T13:56:30.708 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.703+0000 7fb501c53700 1 -- 172.21.15.40:0/3528683078 wait complete. 2020-11-20T13:56:30.712 INFO:teuthology.orchestra.run.smithi040.stdout:55834574853 2020-11-20T13:56:30.713 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.1 flush_pg_stats 2020-11-20T13:56:30.832 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.828+0000 7fd7c40d5700 1 Processor -- start 2020-11-20T13:56:30.832 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.829+0000 7fd7c40d5700 1 -- start start 2020-11-20T13:56:30.832 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.829+0000 7fd7c40d5700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd7bc0b1b60 0x7fd7bc144bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.833 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.829+0000 7fd7c40d5700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7bc0b0d10 con 0x7fd7bc0b6c80 2020-11-20T13:56:30.833 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.829+0000 7fd7c40d5700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7bc0b0fb0 con 0x7fd7bc0b31f0 2020-11-20T13:56:30.833 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.829+0000 7fd7c40d5700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7bc0b7060 con 0x7fd7bc0b1b60 2020-11-20T13:56:30.833 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.829+0000 7fd7c1e71700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd7bc0b6c80 0x7fd7bc0b40f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45290/0 (socket says 172.21.15.40:45290) 2020-11-20T13:56:30.833 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.829+0000 7fd7c1e71700 1 -- 172.21.15.40:0/3866743862 learned_addr learned my addr 172.21.15.40:0/3866743862 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:30.834 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.830+0000 7fd7c2672700 1 --2- 172.21.15.40:0/3866743862 >> v2:172.21.15.116:3300/0 conn(0x7fd7bc0b1b60 0x7fd7bc144bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.834 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.830+0000 7fd7c0e6f700 1 -- 172.21.15.40:0/3866743862 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1925840408 0 0) 0x7fd7bc0b0d10 con 0x7fd7bc0b6c80 2020-11-20T13:56:30.834 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.830+0000 7fd7c0e6f700 1 -- 172.21.15.40:0/3866743862 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7a4003230 con 0x7fd7bc0b6c80 2020-11-20T13:56:30.834 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.830+0000 7fd7c2672700 1 -- 172.21.15.40:0/3866743862 >> v1:172.21.15.116:6789/0 conn(0x7fd7bc0b31f0 legacy=0x7fd7bc0b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.834 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.830+0000 7fd7c0e6f700 1 -- 172.21.15.40:0/3866743862 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2639976575 0 0) 0x7fd7bc0b0fb0 con 0x7fd7bc0b31f0 2020-11-20T13:56:30.835 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.830+0000 7fd7c2672700 1 -- 172.21.15.40:0/3866743862 >> v1:172.21.15.40:6789/0 conn(0x7fd7bc0b6c80 legacy=0x7fd7bc0b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.835 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.830+0000 7fd7c2672700 1 -- 172.21.15.40:0/3866743862 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7bc0b4800 con 0x7fd7bc0b1b60 2020-11-20T13:56:30.835 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.830+0000 7fd7c2672700 1 --2- 172.21.15.40:0/3866743862 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7bc0b1b60 0x7fd7bc144bd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fd7b40032b0 tx=0x7fd7b40091a0).ready entity=mon.1 client_cookie=c8b244af3cb0c5f0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:30.835 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.830+0000 7fd7c0e6f700 1 -- 172.21.15.40:0/3866743862 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7b400b9a0 con 0x7fd7bc0b1b60 2020-11-20T13:56:30.835 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.830+0000 7fd7c0e6f700 1 -- 172.21.15.40:0/3866743862 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b400bb00 con 0x7fd7bc0b1b60 2020-11-20T13:56:30.836 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.830+0000 7fd7c0e6f700 1 -- 172.21.15.40:0/3866743862 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7b400bc70 con 0x7fd7bc0b1b60 2020-11-20T13:56:30.836 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.830+0000 7fd7c40d5700 1 -- 172.21.15.40:0/3866743862 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7bc0b1b60 msgr2=0x7fd7bc144bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.836 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.830+0000 7fd7c40d5700 1 --2- 172.21.15.40:0/3866743862 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7bc0b1b60 0x7fd7bc144bd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fd7b40032b0 tx=0x7fd7b40091a0).stop 2020-11-20T13:56:30.836 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.831+0000 7fd7c40d5700 1 -- 172.21.15.40:0/3866743862 shutdown_connections 2020-11-20T13:56:30.836 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.831+0000 7fd7c40d5700 1 --2- 172.21.15.40:0/3866743862 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7bc0b1b60 0x7fd7bc144bd0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.837 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.831+0000 7fd7c40d5700 1 -- 172.21.15.40:0/3866743862 >> 172.21.15.40:0/3866743862 conn(0x7fd7bc0ba1d0 msgr2=0x7fd7bc0b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:30.837 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.831+0000 7fd7c40d5700 1 -- 172.21.15.40:0/3866743862 shutdown_connections 2020-11-20T13:56:30.837 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.831+0000 7fd7c40d5700 1 -- 172.21.15.40:0/3866743862 wait complete. 2020-11-20T13:56:30.837 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.832+0000 7fd7c40d5700 1 Processor -- start 2020-11-20T13:56:30.838 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.832+0000 7fd7c40d5700 1 -- start start 2020-11-20T13:56:30.840 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.832+0000 7fd7c40d5700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd7bc0b1b60 0x7fd7bc147630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.841 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.832+0000 7fd7c40d5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7bc0b31f0 0x7fd7bc156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.841 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.832+0000 7fd7c40d5700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7bc0b6c80 0x7fd7bc130fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.841 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.832+0000 7fd7c40d5700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc1314d0 con 0x7fd7bc0b31f0 2020-11-20T13:56:30.841 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.832+0000 7fd7c40d5700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc1566d0 con 0x7fd7bc0b6c80 2020-11-20T13:56:30.842 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.833+0000 7fd7c40d5700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc156970 con 0x7fd7bc0b1b60 2020-11-20T13:56:30.842 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.833+0000 7fd7c1670700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7bc0b31f0 0x7fd7bc156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.842 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.833+0000 7fd7c1670700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7bc0b31f0 0x7fd7bc156e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54702/0 (socket says 172.21.15.40:54702) 2020-11-20T13:56:30.843 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.833+0000 7fd7c1670700 1 -- 172.21.15.40:0/15318328 learned_addr learned my addr 172.21.15.40:0/15318328 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:30.843 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.833+0000 7fd7c1e71700 1 --2- 172.21.15.40:0/15318328 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd7bc0b1b60 0x7fd7bc147630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.843 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.833+0000 7fd7c2672700 1 --2- 172.21.15.40:0/15318328 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7bc0b6c80 0x7fd7bc130fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.844 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.833+0000 7fd7c1670700 1 -- 172.21.15.40:0/15318328 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd7bc0b1b60 msgr2=0x7fd7bc147630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.844 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.833+0000 7fd7c1670700 1 --2- 172.21.15.40:0/15318328 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd7bc0b1b60 0x7fd7bc147630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.844 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.833+0000 7fd7c1670700 1 -- 172.21.15.40:0/15318328 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7bc0b6c80 msgr2=0x7fd7bc130fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.844 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.833+0000 7fd7c1670700 1 --2- 172.21.15.40:0/15318328 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7bc0b6c80 0x7fd7bc130fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.845 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.833+0000 7fd7c1670700 1 -- 172.21.15.40:0/15318328 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7bc156380 con 0x7fd7bc0b31f0 2020-11-20T13:56:30.845 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.833+0000 7fd7c2672700 1 --2- 172.21.15.40:0/15318328 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7bc0b6c80 0x7fd7bc130fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:30.845 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.833+0000 7fd7c1670700 1 --2- 172.21.15.40:0/15318328 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7bc0b31f0 0x7fd7bc156e10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fd7b8000b80 tx=0x7fd7b8007960).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:30.845 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.833+0000 7fd7aaffd700 1 -- 172.21.15.40:0/15318328 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7b800e4a0 con 0x7fd7bc0b31f0 2020-11-20T13:56:30.845 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.833+0000 7fd7aaffd700 1 -- 172.21.15.40:0/15318328 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b800e600 con 0x7fd7bc0b31f0 2020-11-20T13:56:30.846 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.833+0000 7fd7aaffd700 1 -- 172.21.15.40:0/15318328 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7b800e960 con 0x7fd7bc0b31f0 2020-11-20T13:56:30.846 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.833+0000 7fd7c40d5700 1 -- 172.21.15.40:0/15318328 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7bc0658c0 con 0x7fd7bc0b31f0 2020-11-20T13:56:30.846 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.833+0000 7fd7c40d5700 1 -- 172.21.15.40:0/15318328 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7bc067230 con 0x7fd7bc0b31f0 2020-11-20T13:56:30.846 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.834+0000 7fd7c40d5700 1 -- 172.21.15.40:0/15318328 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd7bc156b30 con 0x7fd7bc0b31f0 2020-11-20T13:56:30.846 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.834+0000 7fd7aaffd700 1 -- 172.21.15.40:0/15318328 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fd7b8010030 con 0x7fd7bc0b31f0 2020-11-20T13:56:30.847 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.835+0000 7fd7aaffd700 1 --2- 172.21.15.40:0/15318328 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fd79c0513c0 0x7fd79c053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.847 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.835+0000 7fd7aaffd700 1 -- 172.21.15.40:0/15318328 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fd7b8068f50 con 0x7fd7bc0b31f0 2020-11-20T13:56:30.847 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.835+0000 7fd7aaffd700 1 --2- 172.21.15.40:0/15318328 >> [v2:172.21.15.40:6800/22823,v1:172.21.15.40:6801/22823] conn(0x7fd79c058ed0 0x7fd79c05b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.847 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.835+0000 7fd7aaffd700 1 -- 172.21.15.40:0/15318328 --> [v2:172.21.15.40:6800/22823,v1:172.21.15.40:6801/22823] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd79c05b940 con 0x7fd79c058ed0 2020-11-20T13:56:30.847 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.835+0000 7fd7aaffd700 1 -- 172.21.15.40:0/15318328 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7b806b580 con 0x7fd7bc0b31f0 2020-11-20T13:56:30.848 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.835+0000 7fd7c1e71700 1 --2- 172.21.15.40:0/15318328 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fd79c0513c0 0x7fd79c053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.848 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.835+0000 7fd7c2672700 1 --2- 172.21.15.40:0/15318328 >> [v2:172.21.15.40:6800/22823,v1:172.21.15.40:6801/22823] conn(0x7fd79c058ed0 0x7fd79c05b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.848 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.835+0000 7fd7c1e71700 1 --2- 172.21.15.40:0/15318328 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fd79c0513c0 0x7fd79c053880 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fd7ac00d000 tx=0x7fd7ac0026e0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:30.848 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.835+0000 7fd7c2672700 1 --2- 172.21.15.40:0/15318328 >> [v2:172.21.15.40:6800/22823,v1:172.21.15.40:6801/22823] conn(0x7fd79c058ed0 0x7fd79c05b2f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:30.848 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.836+0000 7fd7aaffd700 1 -- 172.21.15.40:0/15318328 <== osd.1 v2:172.21.15.40:6800/22823 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7fd79c05b940 con 0x7fd79c058ed0 2020-11-20T13:56:30.854 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.854+0000 7fd7c40d5700 1 -- 172.21.15.40:0/15318328 --> [v2:172.21.15.40:6800/22823,v1:172.21.15.40:6801/22823] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd7bc057050 con 0x7fd79c058ed0 2020-11-20T13:56:30.855 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.854+0000 7fd7aaffd700 1 -- 172.21.15.40:0/15318328 <== osd.1 v2:172.21.15.40:6800/22823 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd7bc057050 con 0x7fd79c058ed0 2020-11-20T13:56:30.855 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.854+0000 7fd7c40d5700 1 -- 172.21.15.40:0/15318328 >> [v2:172.21.15.40:6800/22823,v1:172.21.15.40:6801/22823] conn(0x7fd79c058ed0 msgr2=0x7fd79c05b2f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.855 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.854+0000 7fd7c40d5700 1 --2- 172.21.15.40:0/15318328 >> [v2:172.21.15.40:6800/22823,v1:172.21.15.40:6801/22823] conn(0x7fd79c058ed0 0x7fd79c05b2f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.856 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.854+0000 7fd7c40d5700 1 -- 172.21.15.40:0/15318328 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fd79c0513c0 msgr2=0x7fd79c053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.856 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.854+0000 7fd7c40d5700 1 --2- 172.21.15.40:0/15318328 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fd79c0513c0 0x7fd79c053880 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fd7ac00d000 tx=0x7fd7ac0026e0).stop 2020-11-20T13:56:30.856 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.854+0000 7fd7c40d5700 1 -- 172.21.15.40:0/15318328 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7bc0b31f0 msgr2=0x7fd7bc156e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.856 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.854+0000 7fd7c40d5700 1 --2- 172.21.15.40:0/15318328 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7bc0b31f0 0x7fd7bc156e10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fd7b8000b80 tx=0x7fd7b8007960).stop 2020-11-20T13:56:30.857 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.854+0000 7fd7c2672700 1 -- 172.21.15.40:0/15318328 reap_dead start 2020-11-20T13:56:30.857 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.855+0000 7fd7c40d5700 1 -- 172.21.15.40:0/15318328 shutdown_connections 2020-11-20T13:56:30.857 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.855+0000 7fd7c40d5700 1 -- 172.21.15.40:0/15318328 >> 172.21.15.40:0/15318328 conn(0x7fd7bc0ba1d0 msgr2=0x7fd7bc0afe20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:30.857 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.855+0000 7fd7c40d5700 1 -- 172.21.15.40:0/15318328 shutdown_connections 2020-11-20T13:56:30.858 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.855+0000 7fd7c40d5700 1 -- 172.21.15.40:0/15318328 wait complete. 2020-11-20T13:56:30.864 INFO:teuthology.orchestra.run.smithi040.stdout:55834574854 2020-11-20T13:56:30.865 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.2 flush_pg_stats 2020-11-20T13:56:30.980 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.980+0000 7f2b7b986700 1 Processor -- start 2020-11-20T13:56:30.980 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.980+0000 7f2b7b986700 1 -- start start 2020-11-20T13:56:30.983 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.980+0000 7f2b7b986700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2b740b6c80 0x7f2b740b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.984 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.981+0000 7f2b7b986700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b740b0d10 con 0x7f2b74144bd0 2020-11-20T13:56:30.984 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.981+0000 7f2b7b986700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b740b0fb0 con 0x7f2b740b1450 2020-11-20T13:56:30.984 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.981+0000 7f2b7b986700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b74145060 con 0x7f2b740b6c80 2020-11-20T13:56:30.984 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.981+0000 7f2b78f21700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2b74144bd0 0x7f2b740b40f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45306/0 (socket says 172.21.15.40:45306) 2020-11-20T13:56:30.984 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.981+0000 7f2b78f21700 1 -- 172.21.15.40:0/77635108 learned_addr learned my addr 172.21.15.40:0/77635108 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:30.985 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.981+0000 7f2b79722700 1 --2- 172.21.15.40:0/77635108 >> v2:172.21.15.116:3300/0 conn(0x7f2b740b6c80 0x7f2b740b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.985 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.981+0000 7f2b73fff700 1 -- 172.21.15.40:0/77635108 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1946197978 0 0) 0x7f2b740b0d10 con 0x7f2b74144bd0 2020-11-20T13:56:30.985 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.981+0000 7f2b73fff700 1 -- 172.21.15.40:0/77635108 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b60003230 con 0x7f2b74144bd0 2020-11-20T13:56:30.985 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.981+0000 7f2b73fff700 1 -- 172.21.15.40:0/77635108 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2213560027 0 0) 0x7f2b60003230 con 0x7f2b74144bd0 2020-11-20T13:56:30.985 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.981+0000 7f2b73fff700 1 -- 172.21.15.40:0/77635108 >> v2:172.21.15.116:3300/0 conn(0x7f2b740b6c80 msgr2=0x7f2b740b31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:30.986 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.981+0000 7f2b73fff700 1 --2- 172.21.15.40:0/77635108 >> v2:172.21.15.116:3300/0 conn(0x7f2b740b6c80 0x7f2b740b31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.986 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.981+0000 7f2b73fff700 1 -- 172.21.15.40:0/77635108 >> v1:172.21.15.116:6789/0 conn(0x7f2b740b1450 legacy=0x7f2b7412ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.986 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.981+0000 7f2b73fff700 1 -- 172.21.15.40:0/77635108 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b740b7060 con 0x7f2b74144bd0 2020-11-20T13:56:30.986 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.981+0000 7f2b73fff700 1 -- 172.21.15.40:0/77635108 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2366654341 0 0) 0x7f2b680016d0 con 0x7f2b74144bd0 2020-11-20T13:56:30.986 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.981+0000 7f2b73fff700 1 -- 172.21.15.40:0/77635108 >> v1:172.21.15.40:6789/0 conn(0x7f2b74144bd0 legacy=0x7f2b740b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.987 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.981+0000 7f2b73fff700 1 --2- 172.21.15.40:0/77635108 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2b60003720 0x7f2b60003b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.987 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.981+0000 7f2b73fff700 1 --2- 172.21.15.40:0/77635108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b600044d0 0x7f2b60004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.987 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.981+0000 7f2b73fff700 1 --2- 172.21.15.40:0/77635108 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b60004e40 0x7f2b6000b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.987 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.981+0000 7f2b73fff700 1 -- 172.21.15.40:0/77635108 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b74145060 con 0x7f2b600044d0 2020-11-20T13:56:30.987 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.981+0000 7f2b73fff700 1 -- 172.21.15.40:0/77635108 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6000bc10 con 0x7f2b60004e40 2020-11-20T13:56:30.988 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.981+0000 7f2b73fff700 1 -- 172.21.15.40:0/77635108 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6000bd50 con 0x7f2b60003720 2020-11-20T13:56:30.988 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.981+0000 7f2b78f21700 1 --2- 172.21.15.40:0/77635108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b600044d0 0x7f2b60004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.988 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.981+0000 7f2b79722700 1 --2- 172.21.15.40:0/77635108 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2b60003720 0x7f2b60003b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.988 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.981+0000 7f2b79f23700 1 --2- 172.21.15.40:0/77635108 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b60004e40 0x7f2b6000b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.988 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.982+0000 7f2b78f21700 1 -- 172.21.15.40:0/77635108 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2b60003720 msgr2=0x7f2b60003b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:30.989 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.982+0000 7f2b78f21700 1 --2- 172.21.15.40:0/77635108 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2b60003720 0x7f2b60003b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.989 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.982+0000 7f2b78f21700 1 -- 172.21.15.40:0/77635108 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b60004e40 msgr2=0x7f2b6000b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:30.989 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.982+0000 7f2b78f21700 1 --2- 172.21.15.40:0/77635108 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b60004e40 0x7f2b6000b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.989 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.982+0000 7f2b79f23700 1 -- 172.21.15.40:0/77635108 reap_dead start 2020-11-20T13:56:30.989 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.982+0000 7f2b78f21700 1 -- 172.21.15.40:0/77635108 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b60003230 con 0x7f2b600044d0 2020-11-20T13:56:30.990 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.982+0000 7f2b78f21700 1 --2- 172.21.15.40:0/77635108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b600044d0 0x7f2b60004950 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f2b680013e0 tx=0x7f2b740b0ea0).ready entity=mon.0 client_cookie=ac89170c066e8bb server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:30.990 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.982+0000 7f2b73fff700 1 -- 172.21.15.40:0/77635108 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b680070d0 con 0x7f2b600044d0 2020-11-20T13:56:30.990 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.982+0000 7f2b73fff700 1 -- 172.21.15.40:0/77635108 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b68007230 con 0x7f2b600044d0 2020-11-20T13:56:30.990 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.982+0000 7f2b73fff700 1 -- 172.21.15.40:0/77635108 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b680055e0 con 0x7f2b600044d0 2020-11-20T13:56:30.991 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.982+0000 7f2b7b986700 1 -- 172.21.15.40:0/77635108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b600044d0 msgr2=0x7f2b60004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.991 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.982+0000 7f2b7b986700 1 --2- 172.21.15.40:0/77635108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b600044d0 0x7f2b60004950 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f2b680013e0 tx=0x7f2b740b0ea0).stop 2020-11-20T13:56:30.991 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.982+0000 7f2b7b986700 1 -- 172.21.15.40:0/77635108 shutdown_connections 2020-11-20T13:56:30.991 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.982+0000 7f2b7b986700 1 --2- 172.21.15.40:0/77635108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b600044d0 0x7f2b60004950 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.991 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.982+0000 7f2b7b986700 1 -- 172.21.15.40:0/77635108 >> 172.21.15.40:0/77635108 conn(0x7f2b740ba1d0 msgr2=0x7f2b740b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:30.992 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.983+0000 7f2b7b986700 1 -- 172.21.15.40:0/77635108 shutdown_connections 2020-11-20T13:56:30.992 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.983+0000 7f2b7b986700 1 -- 172.21.15.40:0/77635108 wait complete. 2020-11-20T13:56:30.992 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.983+0000 7f2b7b986700 1 Processor -- start 2020-11-20T13:56:30.992 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.983+0000 7f2b7b986700 1 -- start start 2020-11-20T13:56:30.995 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.984+0000 7f2b7b986700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2b600044d0 0x7f2b74067000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.995 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.984+0000 7f2b7b986700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b741433b0 0x7f2b74159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.995 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.984+0000 7f2b7b986700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b740ae9d0 0x7f2b741291b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.996 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.984+0000 7f2b7b986700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b74143b60 con 0x7f2b741433b0 2020-11-20T13:56:30.996 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.984+0000 7f2b7b986700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7415a4d0 con 0x7f2b740ae9d0 2020-11-20T13:56:30.996 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.984+0000 7f2b7b986700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b741438c0 con 0x7f2b600044d0 2020-11-20T13:56:30.996 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.984+0000 7f2b78f21700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b741433b0 0x7f2b74159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.996 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.984+0000 7f2b78f21700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b741433b0 0x7f2b74159fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54724/0 (socket says 172.21.15.40:54724) 2020-11-20T13:56:30.997 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.984+0000 7f2b78f21700 1 -- 172.21.15.40:0/2085457908 learned_addr learned my addr 172.21.15.40:0/2085457908 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:30.997 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.984+0000 7f2b79722700 1 --2- 172.21.15.40:0/2085457908 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2b600044d0 0x7f2b74067000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.997 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.984+0000 7f2b79f23700 1 --2- 172.21.15.40:0/2085457908 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b740ae9d0 0x7f2b741291b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.997 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.984+0000 7f2b78f21700 1 -- 172.21.15.40:0/2085457908 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2b600044d0 msgr2=0x7f2b74067000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.997 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.985+0000 7f2b78f21700 1 --2- 172.21.15.40:0/2085457908 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2b600044d0 0x7f2b74067000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.998 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.985+0000 7f2b78f21700 1 -- 172.21.15.40:0/2085457908 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b740ae9d0 msgr2=0x7f2b741291b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.998 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.985+0000 7f2b78f21700 1 --2- 172.21.15.40:0/2085457908 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b740ae9d0 0x7f2b741291b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.998 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.985+0000 7f2b78f21700 1 -- 172.21.15.40:0/2085457908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b7412ecd0 con 0x7f2b741433b0 2020-11-20T13:56:30.998 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.985+0000 7f2b78f21700 1 --2- 172.21.15.40:0/2085457908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b741433b0 0x7f2b74159fe0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f2b680013e0 tx=0x7f2b68015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:30.998 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.985+0000 7f2b727fc700 1 -- 172.21.15.40:0/2085457908 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b68005e70 con 0x7f2b741433b0 2020-11-20T13:56:30.999 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.985+0000 7f2b727fc700 1 -- 172.21.15.40:0/2085457908 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b68013660 con 0x7f2b741433b0 2020-11-20T13:56:30.999 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.985+0000 7f2b727fc700 1 -- 172.21.15.40:0/2085457908 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b680139c0 con 0x7f2b741433b0 2020-11-20T13:56:30.999 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.985+0000 7f2b7b986700 1 -- 172.21.15.40:0/2085457908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b740aee40 con 0x7f2b741433b0 2020-11-20T13:56:30.999 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.985+0000 7f2b7b986700 1 -- 172.21.15.40:0/2085457908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b741298e0 con 0x7f2b741433b0 2020-11-20T13:56:30.999 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.986+0000 7f2b7b986700 1 -- 172.21.15.40:0/2085457908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2b7412eb50 con 0x7f2b741433b0 2020-11-20T13:56:31.000 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.986+0000 7f2b727fc700 1 -- 172.21.15.40:0/2085457908 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f2b68026020 con 0x7f2b741433b0 2020-11-20T13:56:31.000 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.986+0000 7f2b727fc700 1 --2- 172.21.15.40:0/2085457908 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f2b54051290 0x7f2b54053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.000 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.986+0000 7f2b727fc700 1 -- 172.21.15.40:0/2085457908 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f2b6806a750 con 0x7f2b741433b0 2020-11-20T13:56:31.000 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.986+0000 7f2b727fc700 1 --2- 172.21.15.40:0/2085457908 >> [v2:172.21.15.40:6808/22824,v1:172.21.15.40:6809/22824] conn(0x7f2b54058da0 0x7f2b5405b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.000 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.986+0000 7f2b727fc700 1 -- 172.21.15.40:0/2085457908 --> [v2:172.21.15.40:6808/22824,v1:172.21.15.40:6809/22824] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2b5405b810 con 0x7f2b54058da0 2020-11-20T13:56:31.001 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.986+0000 7f2b727fc700 1 -- 172.21.15.40:0/2085457908 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b6806cde0 con 0x7f2b741433b0 2020-11-20T13:56:31.001 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.986+0000 7f2b79722700 1 --2- 172.21.15.40:0/2085457908 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f2b54051290 0x7f2b54053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.001 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.987+0000 7f2b79f23700 1 --2- 172.21.15.40:0/2085457908 >> [v2:172.21.15.40:6808/22824,v1:172.21.15.40:6809/22824] conn(0x7f2b54058da0 0x7f2b5405b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.001 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.987+0000 7f2b79722700 1 --2- 172.21.15.40:0/2085457908 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f2b54051290 0x7f2b54053750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f2b6c00b000 tx=0x7f2b6c0091a0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:31.001 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.987+0000 7f2b79f23700 1 --2- 172.21.15.40:0/2085457908 >> [v2:172.21.15.40:6808/22824,v1:172.21.15.40:6809/22824] conn(0x7f2b54058da0 0x7f2b5405b1c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:31.002 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:30.988+0000 7f2b727fc700 1 -- 172.21.15.40:0/2085457908 <== osd.2 v2:172.21.15.40:6808/22824 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7f2b5405b810 con 0x7f2b54058da0 2020-11-20T13:56:31.005 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.005+0000 7f2b7b986700 1 -- 172.21.15.40:0/2085457908 --> [v2:172.21.15.40:6808/22824,v1:172.21.15.40:6809/22824] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2b7412ee60 con 0x7f2b54058da0 2020-11-20T13:56:31.007 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.006+0000 7f2b727fc700 1 -- 172.21.15.40:0/2085457908 <== osd.2 v2:172.21.15.40:6808/22824 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2b7412ee60 con 0x7f2b54058da0 2020-11-20T13:56:31.007 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.006+0000 7f2b7b986700 1 -- 172.21.15.40:0/2085457908 >> [v2:172.21.15.40:6808/22824,v1:172.21.15.40:6809/22824] conn(0x7f2b54058da0 msgr2=0x7f2b5405b1c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.007 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.006+0000 7f2b7b986700 1 --2- 172.21.15.40:0/2085457908 >> [v2:172.21.15.40:6808/22824,v1:172.21.15.40:6809/22824] conn(0x7f2b54058da0 0x7f2b5405b1c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.007 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.006+0000 7f2b7b986700 1 -- 172.21.15.40:0/2085457908 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f2b54051290 msgr2=0x7f2b54053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.007 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.006+0000 7f2b7b986700 1 --2- 172.21.15.40:0/2085457908 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f2b54051290 0x7f2b54053750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f2b6c00b000 tx=0x7f2b6c0091a0).stop 2020-11-20T13:56:31.008 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.006+0000 7f2b7b986700 1 -- 172.21.15.40:0/2085457908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b741433b0 msgr2=0x7f2b74159fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.008 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.006+0000 7f2b7b986700 1 --2- 172.21.15.40:0/2085457908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b741433b0 0x7f2b74159fe0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f2b680013e0 tx=0x7f2b68015000).stop 2020-11-20T13:56:31.008 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.006+0000 7f2b79f23700 1 -- 172.21.15.40:0/2085457908 reap_dead start 2020-11-20T13:56:31.009 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.006+0000 7f2b7b986700 1 -- 172.21.15.40:0/2085457908 shutdown_connections 2020-11-20T13:56:31.009 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.006+0000 7f2b7b986700 1 -- 172.21.15.40:0/2085457908 >> 172.21.15.40:0/2085457908 conn(0x7f2b740ba1d0 msgr2=0x7f2b740b4450 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:31.009 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.006+0000 7f2b7b986700 1 -- 172.21.15.40:0/2085457908 shutdown_connections 2020-11-20T13:56:31.009 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.006+0000 7f2b7b986700 1 -- 172.21.15.40:0/2085457908 wait complete. 2020-11-20T13:56:31.017 INFO:teuthology.orchestra.run.smithi040.stdout:55834574853 2020-11-20T13:56:31.018 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.3 flush_pg_stats 2020-11-20T13:56:31.133 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.133+0000 7fa2a2798700 1 Processor -- start 2020-11-20T13:56:31.134 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.133+0000 7fa2a2798700 1 -- start start 2020-11-20T13:56:31.136 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.134+0000 7fa2a2798700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa29c061f90 0x7fa29c14b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.136 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.134+0000 7fa2a2798700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa29c0b6980 0x7fa29c0b6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.136 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.134+0000 7fa2a2798700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa29c061310 con 0x7fa29c05c510 2020-11-20T13:56:31.137 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.134+0000 7fa2a2798700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa29c0615b0 con 0x7fa29c0b6980 2020-11-20T13:56:31.137 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.134+0000 7fa2a2798700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa29c061850 con 0x7fa29c061f90 2020-11-20T13:56:31.137 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.134+0000 7fa2a0d35700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa29c0b6980 0x7fa29c0b6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.137 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.134+0000 7fa2a0d35700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa29c0b6980 0x7fa29c0b6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54736/0 (socket says 172.21.15.40:54736) 2020-11-20T13:56:31.138 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.134+0000 7fa2a0d35700 1 -- 172.21.15.40:0/3552586689 learned_addr learned my addr 172.21.15.40:0/3552586689 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:31.138 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.134+0000 7fa29b7fe700 1 --2- 172.21.15.40:0/3552586689 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa29c061f90 0x7fa29c14b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.138 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.134+0000 7fa2a0d35700 1 -- 172.21.15.40:0/3552586689 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa29c061f90 msgr2=0x7fa29c14b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:31.138 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.134+0000 7fa2a0d35700 1 --2- 172.21.15.40:0/3552586689 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa29c061f90 0x7fa29c14b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.138 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.134+0000 7fa2a0d35700 1 -- 172.21.15.40:0/3552586689 >> v1:172.21.15.116:6789/0 conn(0x7fa29c05c510 legacy=0x7fa29c12d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:31.139 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.134+0000 7fa2a0d35700 1 -- 172.21.15.40:0/3552586689 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa29c061af0 con 0x7fa29c0b6980 2020-11-20T13:56:31.139 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.134+0000 7fa29b7fe700 1 --2- 172.21.15.40:0/3552586689 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa29c061f90 0x7fa29c14b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:31.139 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.134+0000 7fa2a0d35700 1 --2- 172.21.15.40:0/3552586689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa29c0b6980 0x7fa29c0b6df0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fa29c062370 tx=0x7fa294009270).ready entity=mon.0 client_cookie=4d558ae920d0b846 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:31.139 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.134+0000 7fa29affd700 1 -- 172.21.15.40:0/3552586689 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa29400bbc0 con 0x7fa29c0b6980 2020-11-20T13:56:31.140 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.135+0000 7fa29affd700 1 -- 172.21.15.40:0/3552586689 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa29400bd20 con 0x7fa29c0b6980 2020-11-20T13:56:31.140 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.135+0000 7fa29affd700 1 -- 172.21.15.40:0/3552586689 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa294011400 con 0x7fa29c0b6980 2020-11-20T13:56:31.140 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.135+0000 7fa2a2798700 1 -- 172.21.15.40:0/3552586689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa29c0b6980 msgr2=0x7fa29c0b6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.140 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.135+0000 7fa2a2798700 1 --2- 172.21.15.40:0/3552586689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa29c0b6980 0x7fa29c0b6df0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fa29c062370 tx=0x7fa294009270).stop 2020-11-20T13:56:31.140 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.135+0000 7fa2a2798700 1 -- 172.21.15.40:0/3552586689 shutdown_connections 2020-11-20T13:56:31.141 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.135+0000 7fa2a2798700 1 --2- 172.21.15.40:0/3552586689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa29c0b6980 0x7fa29c0b6df0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.141 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.135+0000 7fa2a2798700 1 --2- 172.21.15.40:0/3552586689 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa29c061f90 0x7fa29c14b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.141 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.135+0000 7fa2a2798700 1 -- 172.21.15.40:0/3552586689 >> 172.21.15.40:0/3552586689 conn(0x7fa29c0bdb70 msgr2=0x7fa29c0bdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:31.141 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.135+0000 7fa2a2798700 1 -- 172.21.15.40:0/3552586689 shutdown_connections 2020-11-20T13:56:31.142 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.135+0000 7fa2a2798700 1 -- 172.21.15.40:0/3552586689 wait complete. 2020-11-20T13:56:31.142 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.136+0000 7fa2a2798700 1 Processor -- start 2020-11-20T13:56:31.142 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.136+0000 7fa2a2798700 1 -- start start 2020-11-20T13:56:31.147 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.137+0000 7fa2a2798700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa29c05c510 0x7fa29c147960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.147 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.137+0000 7fa2a2798700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa29c061f90 0x7fa29c0b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.147 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.137+0000 7fa2a2798700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa29c0b6980 0x7fa29c0accf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.147 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.137+0000 7fa2a2798700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa29c0b0580 con 0x7fa29c05c510 2020-11-20T13:56:31.148 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.137+0000 7fa2a2798700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa29c0b0820 con 0x7fa29c061f90 2020-11-20T13:56:31.148 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.137+0000 7fa2a2798700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa29c0b0ac0 con 0x7fa29c0b6980 2020-11-20T13:56:31.148 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.137+0000 7fa29bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa29c05c510 0x7fa29c147960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.148 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.137+0000 7fa29bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa29c05c510 0x7fa29c147960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54738/0 (socket says 172.21.15.40:54738) 2020-11-20T13:56:31.149 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.137+0000 7fa29bfff700 1 -- 172.21.15.40:0/4244157942 learned_addr learned my addr 172.21.15.40:0/4244157942 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:31.149 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.137+0000 7fa2a0d35700 1 --2- 172.21.15.40:0/4244157942 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa29c0b6980 0x7fa29c0accf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.149 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.137+0000 7fa29b7fe700 1 --2- 172.21.15.40:0/4244157942 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa29c061f90 0x7fa29c0b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.149 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.137+0000 7fa29bfff700 1 -- 172.21.15.40:0/4244157942 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa29c0b6980 msgr2=0x7fa29c0accf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.149 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.137+0000 7fa29bfff700 1 --2- 172.21.15.40:0/4244157942 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa29c0b6980 0x7fa29c0accf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.150 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.137+0000 7fa29bfff700 1 -- 172.21.15.40:0/4244157942 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa29c061f90 msgr2=0x7fa29c0b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.150 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.137+0000 7fa29bfff700 1 --2- 172.21.15.40:0/4244157942 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa29c061f90 0x7fa29c0b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.150 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.137+0000 7fa29bfff700 1 -- 172.21.15.40:0/4244157942 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa29c0b1450 con 0x7fa29c05c510 2020-11-20T13:56:31.150 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.137+0000 7fa29bfff700 1 --2- 172.21.15.40:0/4244157942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa29c05c510 0x7fa29c147960 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fa28c001f50 tx=0x7fa28c007b70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:31.151 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.137+0000 7fa2997fa700 1 -- 172.21.15.40:0/4244157942 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa28c00e4a0 con 0x7fa29c05c510 2020-11-20T13:56:31.151 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.138+0000 7fa2997fa700 1 -- 172.21.15.40:0/4244157942 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa28c00e600 con 0x7fa29c05c510 2020-11-20T13:56:31.151 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.138+0000 7fa2a2798700 1 -- 172.21.15.40:0/4244157942 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa29c0b15e0 con 0x7fa29c05c510 2020-11-20T13:56:31.151 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.138+0000 7fa2997fa700 1 -- 172.21.15.40:0/4244157942 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa28c00e960 con 0x7fa29c05c510 2020-11-20T13:56:31.151 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.138+0000 7fa2a2798700 1 -- 172.21.15.40:0/4244157942 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa29c0aee00 con 0x7fa29c05c510 2020-11-20T13:56:31.152 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.138+0000 7fa2a2798700 1 -- 172.21.15.40:0/4244157942 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa29c057050 con 0x7fa29c05c510 2020-11-20T13:56:31.153 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.139+0000 7fa2997fa700 1 -- 172.21.15.40:0/4244157942 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fa28c019030 con 0x7fa29c05c510 2020-11-20T13:56:31.153 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.139+0000 7fa2997fa700 1 --2- 172.21.15.40:0/4244157942 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa27c051210 0x7fa27c0536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.153 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.139+0000 7fa2997fa700 1 -- 172.21.15.40:0/4244157942 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fa28c068480 con 0x7fa29c05c510 2020-11-20T13:56:31.153 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.139+0000 7fa29b7fe700 1 --2- 172.21.15.40:0/4244157942 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa27c051210 0x7fa27c0536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.154 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.139+0000 7fa2997fa700 1 --2- 172.21.15.40:0/4244157942 >> [v2:172.21.15.40:6817/22826,v1:172.21.15.40:6819/22826] conn(0x7fa27c058d20 0x7fa27c05b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.154 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.139+0000 7fa2997fa700 1 -- 172.21.15.40:0/4244157942 --> [v2:172.21.15.40:6817/22826,v1:172.21.15.40:6819/22826] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa27c05b790 con 0x7fa27c058d20 2020-11-20T13:56:31.154 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.139+0000 7fa2997fa700 1 -- 172.21.15.40:0/4244157942 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa28c06aa90 con 0x7fa29c05c510 2020-11-20T13:56:31.154 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.139+0000 7fa2a0d35700 1 --2- 172.21.15.40:0/4244157942 >> [v2:172.21.15.40:6817/22826,v1:172.21.15.40:6819/22826] conn(0x7fa27c058d20 0x7fa27c05b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.154 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.139+0000 7fa29b7fe700 1 --2- 172.21.15.40:0/4244157942 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa27c051210 0x7fa27c0536d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fa29000b000 tx=0x7fa290008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:31.155 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.140+0000 7fa2a0d35700 1 --2- 172.21.15.40:0/4244157942 >> [v2:172.21.15.40:6817/22826,v1:172.21.15.40:6819/22826] conn(0x7fa27c058d20 0x7fa27c05b140 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:31.155 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.141+0000 7fa2997fa700 1 -- 172.21.15.40:0/4244157942 <== osd.3 v2:172.21.15.40:6817/22826 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7fa27c05b790 con 0x7fa27c058d20 2020-11-20T13:56:31.158 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.158+0000 7fa2a2798700 1 -- 172.21.15.40:0/4244157942 --> [v2:172.21.15.40:6817/22826,v1:172.21.15.40:6819/22826] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa29c0be680 con 0x7fa27c058d20 2020-11-20T13:56:31.159 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.159+0000 7fa2997fa700 1 -- 172.21.15.40:0/4244157942 <== osd.3 v2:172.21.15.40:6817/22826 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa29c0be680 con 0x7fa27c058d20 2020-11-20T13:56:31.160 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.159+0000 7fa2a2798700 1 -- 172.21.15.40:0/4244157942 >> [v2:172.21.15.40:6817/22826,v1:172.21.15.40:6819/22826] conn(0x7fa27c058d20 msgr2=0x7fa27c05b140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.160 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.159+0000 7fa2a2798700 1 --2- 172.21.15.40:0/4244157942 >> [v2:172.21.15.40:6817/22826,v1:172.21.15.40:6819/22826] conn(0x7fa27c058d20 0x7fa27c05b140 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.160 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.159+0000 7fa2a2798700 1 -- 172.21.15.40:0/4244157942 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa27c051210 msgr2=0x7fa27c0536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.160 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.159+0000 7fa2a2798700 1 --2- 172.21.15.40:0/4244157942 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa27c051210 0x7fa27c0536d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fa29000b000 tx=0x7fa290008040).stop 2020-11-20T13:56:31.161 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.159+0000 7fa2a2798700 1 -- 172.21.15.40:0/4244157942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa29c05c510 msgr2=0x7fa29c147960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.161 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.159+0000 7fa2a2798700 1 --2- 172.21.15.40:0/4244157942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa29c05c510 0x7fa29c147960 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fa28c001f50 tx=0x7fa28c007b70).stop 2020-11-20T13:56:31.161 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.159+0000 7fa2a0d35700 1 -- 172.21.15.40:0/4244157942 reap_dead start 2020-11-20T13:56:31.161 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.159+0000 7fa2a2798700 1 -- 172.21.15.40:0/4244157942 shutdown_connections 2020-11-20T13:56:31.161 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.159+0000 7fa2a2798700 1 -- 172.21.15.40:0/4244157942 >> 172.21.15.40:0/4244157942 conn(0x7fa29c0bdb70 msgr2=0x7fa29c0bc810 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:31.162 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.159+0000 7fa2a2798700 1 -- 172.21.15.40:0/4244157942 shutdown_connections 2020-11-20T13:56:31.162 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.160+0000 7fa2a2798700 1 -- 172.21.15.40:0/4244157942 wait complete. 2020-11-20T13:56:31.169 INFO:teuthology.orchestra.run.smithi040.stdout:55834574854 2020-11-20T13:56:31.170 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.4 flush_pg_stats 2020-11-20T13:56:31.285 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.284+0000 7f4225291700 1 Processor -- start 2020-11-20T13:56:31.285 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.284+0000 7f4225291700 1 -- start start 2020-11-20T13:56:31.287 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.285+0000 7f4225291700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f42200b3960 0x7f422005faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.287 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.285+0000 7f4225291700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4220144390 0x7f422014cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.287 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.285+0000 7f4225291700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42200b5fd0 con 0x7f42200b5bf0 2020-11-20T13:56:31.287 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.285+0000 7f4225291700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42200b05f0 con 0x7f4220144390 2020-11-20T13:56:31.287 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.285+0000 7f4225291700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4220144870 con 0x7f42200b3960 2020-11-20T13:56:31.288 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.286+0000 7f421e7fc700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f42200b3960 0x7f422005faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.288 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.286+0000 7f421f7fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4220144390 0x7f422014cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.288 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.286+0000 7f421e7fc700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f42200b3960 0x7f422005faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.40:39824/0 (socket says 172.21.15.40:39824) 2020-11-20T13:56:31.288 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.286+0000 7f421e7fc700 1 -- 172.21.15.40:0/1917133007 learned_addr learned my addr 172.21.15.40:0/1917133007 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:31.289 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.286+0000 7f421f7fe700 1 -- 172.21.15.40:0/1917133007 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f42200b3960 msgr2=0x7f422005faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:31.289 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.286+0000 7f421f7fe700 1 --2- 172.21.15.40:0/1917133007 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f42200b3960 0x7f422005faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.289 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.286+0000 7f421f7fe700 1 -- 172.21.15.40:0/1917133007 >> v1:172.21.15.116:6789/0 conn(0x7f42200b5bf0 legacy=0x7f422005f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.289 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.286+0000 7f421f7fe700 1 -- 172.21.15.40:0/1917133007 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42200b0c30 con 0x7f4220144390 2020-11-20T13:56:31.289 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.286+0000 7f421e7fc700 1 --2- 172.21.15.40:0/1917133007 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f42200b3960 0x7f422005faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:56:31.290 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.286+0000 7f421f7fe700 1 --2- 172.21.15.40:0/1917133007 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4220144390 0x7f422014cf50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f4208001120 tx=0x7f4218006e50).ready entity=mon.1 client_cookie=5ff93d98c8928245 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:31.290 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.286+0000 7f421dffb700 1 -- 172.21.15.40:0/1917133007 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4218014070 con 0x7f4220144390 2020-11-20T13:56:31.290 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.286+0000 7f421dffb700 1 -- 172.21.15.40:0/1917133007 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f421800a940 con 0x7f4220144390 2020-11-20T13:56:31.290 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.286+0000 7f421dffb700 1 -- 172.21.15.40:0/1917133007 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f421800aca0 con 0x7f4220144390 2020-11-20T13:56:31.290 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.286+0000 7f4225291700 1 -- 172.21.15.40:0/1917133007 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4220144390 msgr2=0x7f422014cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.291 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.286+0000 7f4225291700 1 --2- 172.21.15.40:0/1917133007 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4220144390 0x7f422014cf50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f4208001120 tx=0x7f4218006e50).stop 2020-11-20T13:56:31.294 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.287+0000 7f4225291700 1 -- 172.21.15.40:0/1917133007 shutdown_connections 2020-11-20T13:56:31.294 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.287+0000 7f4225291700 1 --2- 172.21.15.40:0/1917133007 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4220144390 0x7f422014cf50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.294 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.287+0000 7f4225291700 1 --2- 172.21.15.40:0/1917133007 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f42200b3960 0x7f422005faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.294 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.287+0000 7f4225291700 1 -- 172.21.15.40:0/1917133007 >> 172.21.15.40:0/1917133007 conn(0x7f42200ba1d0 msgr2=0x7f42200b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:31.295 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.287+0000 7f4225291700 1 -- 172.21.15.40:0/1917133007 shutdown_connections 2020-11-20T13:56:31.295 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.287+0000 7f4225291700 1 -- 172.21.15.40:0/1917133007 wait complete. 2020-11-20T13:56:31.295 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.288+0000 7f4225291700 1 Processor -- start 2020-11-20T13:56:31.295 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.288+0000 7f4225291700 1 -- start start 2020-11-20T13:56:31.295 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.289+0000 7f4225291700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f42200b3960 0x7f4220065770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.296 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.289+0000 7f4225291700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42200b5bf0 0x7f4220157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.296 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.289+0000 7f4225291700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4220144390 0x7f42201400c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.296 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.289+0000 7f4225291700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422013ab70 con 0x7f42200b5bf0 2020-11-20T13:56:31.296 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.289+0000 7f4225291700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42200b7650 con 0x7f4220144390 2020-11-20T13:56:31.296 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.289+0000 7f4225291700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42200b78f0 con 0x7f42200b3960 2020-11-20T13:56:31.297 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.289+0000 7f421e7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42200b5bf0 0x7f4220157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.297 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.289+0000 7f421e7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42200b5bf0 0x7f4220157520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54756/0 (socket says 172.21.15.40:54756) 2020-11-20T13:56:31.297 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.289+0000 7f421e7fc700 1 -- 172.21.15.40:0/1432245494 learned_addr learned my addr 172.21.15.40:0/1432245494 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:31.298 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.289+0000 7f421f7fe700 1 --2- 172.21.15.40:0/1432245494 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4220144390 0x7f42201400c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.298 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.289+0000 7f421effd700 1 --2- 172.21.15.40:0/1432245494 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f42200b3960 0x7f4220065770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.298 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.289+0000 7f421e7fc700 1 -- 172.21.15.40:0/1432245494 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f42200b3960 msgr2=0x7f4220065770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.298 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.289+0000 7f421e7fc700 1 --2- 172.21.15.40:0/1432245494 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f42200b3960 0x7f4220065770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.299 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.289+0000 7f421e7fc700 1 -- 172.21.15.40:0/1432245494 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4220144390 msgr2=0x7f42201400c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.299 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.289+0000 7f421e7fc700 1 --2- 172.21.15.40:0/1432245494 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4220144390 0x7f42201400c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.299 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.289+0000 7f421e7fc700 1 -- 172.21.15.40:0/1432245494 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42200b7b90 con 0x7f42200b5bf0 2020-11-20T13:56:31.299 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.289+0000 7f421effd700 1 --2- 172.21.15.40:0/1432245494 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f42200b3960 0x7f4220065770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:31.299 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.289+0000 7f421e7fc700 1 --2- 172.21.15.40:0/1432245494 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42200b5bf0 0x7f4220157520 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f421000c000 tx=0x7f4210009270).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:31.300 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.289+0000 7f4217fff700 1 -- 172.21.15.40:0/1432245494 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f421000cc30 con 0x7f42200b5bf0 2020-11-20T13:56:31.300 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.289+0000 7f4217fff700 1 -- 172.21.15.40:0/1432245494 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f421000cd90 con 0x7f42200b5bf0 2020-11-20T13:56:31.300 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.289+0000 7f4225291700 1 -- 172.21.15.40:0/1432245494 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42200b7e90 con 0x7f42200b5bf0 2020-11-20T13:56:31.300 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.289+0000 7f4217fff700 1 -- 172.21.15.40:0/1432245494 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f42100125e0 con 0x7f42200b5bf0 2020-11-20T13:56:31.300 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.290+0000 7f4225291700 1 -- 172.21.15.40:0/1432245494 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4220157a10 con 0x7f42200b5bf0 2020-11-20T13:56:31.301 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.290+0000 7f4225291700 1 -- 172.21.15.40:0/1432245494 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f42201580a0 con 0x7f42200b5bf0 2020-11-20T13:56:31.301 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.291+0000 7f4217fff700 1 -- 172.21.15.40:0/1432245494 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f4210012740 con 0x7f42200b5bf0 2020-11-20T13:56:31.301 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.291+0000 7f4217fff700 1 --2- 172.21.15.40:0/1432245494 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f4204051210 0x7f42040536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.301 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.291+0000 7f4217fff700 1 -- 172.21.15.40:0/1432245494 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f4210069af0 con 0x7f42200b5bf0 2020-11-20T13:56:31.301 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.291+0000 7f4217fff700 1 --2- 172.21.15.40:0/1432245494 >> [v2:172.21.15.116:6818/23181,v1:172.21.15.116:6819/23181] conn(0x7f4204058d20 0x7f420405b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.302 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.291+0000 7f4217fff700 1 -- 172.21.15.40:0/1432245494 --> [v2:172.21.15.116:6818/23181,v1:172.21.15.116:6819/23181] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f420405b790 con 0x7f4204058d20 2020-11-20T13:56:31.302 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.291+0000 7f4217fff700 1 -- 172.21.15.40:0/1432245494 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f421006e240 con 0x7f42200b5bf0 2020-11-20T13:56:31.302 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.291+0000 7f421effd700 1 --2- 172.21.15.40:0/1432245494 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f4204051210 0x7f42040536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.302 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.291+0000 7f421f7fe700 1 --2- 172.21.15.40:0/1432245494 >> [v2:172.21.15.116:6818/23181,v1:172.21.15.116:6819/23181] conn(0x7f4204058d20 0x7f420405b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.302 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.291+0000 7f421effd700 1 --2- 172.21.15.40:0/1432245494 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f4204051210 0x7f42040536d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f420800b000 tx=0x7f4208009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:31.303 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.292+0000 7f421f7fe700 1 --2- 172.21.15.40:0/1432245494 >> [v2:172.21.15.116:6818/23181,v1:172.21.15.116:6819/23181] conn(0x7f4204058d20 0x7f420405b140 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:31.303 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.292+0000 7f4217fff700 1 -- 172.21.15.40:0/1432245494 <== osd.4 v2:172.21.15.116:6818/23181 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7f420405b790 con 0x7f4204058d20 2020-11-20T13:56:31.311 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.310+0000 7f4225291700 1 -- 172.21.15.40:0/1432245494 --> [v2:172.21.15.116:6818/23181,v1:172.21.15.116:6819/23181] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4220157d20 con 0x7f4204058d20 2020-11-20T13:56:31.312 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.311+0000 7f4217fff700 1 -- 172.21.15.40:0/1432245494 <== osd.4 v2:172.21.15.116:6818/23181 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f421800c7b0 con 0x7f4204058d20 2020-11-20T13:56:31.312 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.311+0000 7f4225291700 1 -- 172.21.15.40:0/1432245494 >> [v2:172.21.15.116:6818/23181,v1:172.21.15.116:6819/23181] conn(0x7f4204058d20 msgr2=0x7f420405b140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.312 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.311+0000 7f4225291700 1 --2- 172.21.15.40:0/1432245494 >> [v2:172.21.15.116:6818/23181,v1:172.21.15.116:6819/23181] conn(0x7f4204058d20 0x7f420405b140 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.312 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.311+0000 7f4225291700 1 -- 172.21.15.40:0/1432245494 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f4204051210 msgr2=0x7f42040536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.313 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.311+0000 7f4225291700 1 --2- 172.21.15.40:0/1432245494 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f4204051210 0x7f42040536d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f420800b000 tx=0x7f4208009040).stop 2020-11-20T13:56:31.314 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.311+0000 7f4225291700 1 -- 172.21.15.40:0/1432245494 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42200b5bf0 msgr2=0x7f4220157520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.315 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.311+0000 7f4225291700 1 --2- 172.21.15.40:0/1432245494 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42200b5bf0 0x7f4220157520 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f421000c000 tx=0x7f4210009270).stop 2020-11-20T13:56:31.315 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.311+0000 7f421f7fe700 1 -- 172.21.15.40:0/1432245494 reap_dead start 2020-11-20T13:56:31.315 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.311+0000 7f4225291700 1 -- 172.21.15.40:0/1432245494 shutdown_connections 2020-11-20T13:56:31.315 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.311+0000 7f4225291700 1 -- 172.21.15.40:0/1432245494 >> 172.21.15.40:0/1432245494 conn(0x7f42200ba1d0 msgr2=0x7f42200b28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:31.315 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.311+0000 7f4225291700 1 -- 172.21.15.40:0/1432245494 shutdown_connections 2020-11-20T13:56:31.316 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.311+0000 7f4225291700 1 -- 172.21.15.40:0/1432245494 wait complete. 2020-11-20T13:56:31.321 INFO:teuthology.orchestra.run.smithi040.stdout:55834574853 2020-11-20T13:56:31.322 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.5 flush_pg_stats 2020-11-20T13:56:31.437 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.437+0000 7fbf80b6f700 1 Processor -- start 2020-11-20T13:56:31.438 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.437+0000 7fbf80b6f700 1 -- start start 2020-11-20T13:56:31.440 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.438+0000 7fbf80b6f700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fbf7c0b1b60 0x7fbf7c144bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.440 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.438+0000 7fbf80b6f700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf7c0b0d10 con 0x7fbf7c0b6c80 2020-11-20T13:56:31.440 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.438+0000 7fbf80b6f700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf7c0b0fb0 con 0x7fbf7c0b31f0 2020-11-20T13:56:31.440 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.438+0000 7fbf80b6f700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf7c0b7060 con 0x7fbf7c0b1b60 2020-11-20T13:56:31.441 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.438+0000 7fbf7a59c700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fbf7c0b6c80 0x7fbf7c0b40f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45360/0 (socket says 172.21.15.40:45360) 2020-11-20T13:56:31.441 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.438+0000 7fbf7a59c700 1 -- 172.21.15.40:0/3044955964 learned_addr learned my addr 172.21.15.40:0/3044955964 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:31.441 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.438+0000 7fbf7ad9d700 1 --2- 172.21.15.40:0/3044955964 >> v2:172.21.15.116:3300/0 conn(0x7fbf7c0b1b60 0x7fbf7c144bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.441 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.439+0000 7fbf7959a700 1 -- 172.21.15.40:0/3044955964 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2340293248 0 0) 0x7fbf7c0b0d10 con 0x7fbf7c0b6c80 2020-11-20T13:56:31.441 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.439+0000 7fbf7959a700 1 -- 172.21.15.40:0/3044955964 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf60003230 con 0x7fbf7c0b6c80 2020-11-20T13:56:31.442 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.439+0000 7fbf7ad9d700 1 -- 172.21.15.40:0/3044955964 >> v1:172.21.15.116:6789/0 conn(0x7fbf7c0b31f0 legacy=0x7fbf7c0b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.442 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.439+0000 7fbf7ad9d700 1 -- 172.21.15.40:0/3044955964 >> v1:172.21.15.40:6789/0 conn(0x7fbf7c0b6c80 legacy=0x7fbf7c0b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.442 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.439+0000 7fbf7959a700 1 -- 172.21.15.40:0/3044955964 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1891911891 0 0) 0x7fbf7c0b0fb0 con 0x7fbf7c0b31f0 2020-11-20T13:56:31.442 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.439+0000 7fbf7ad9d700 1 -- 172.21.15.40:0/3044955964 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf7c0b4800 con 0x7fbf7c0b1b60 2020-11-20T13:56:31.442 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.439+0000 7fbf7ad9d700 1 --2- 172.21.15.40:0/3044955964 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf7c0b1b60 0x7fbf7c144bd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fbf740032b0 tx=0x7fbf740091a0).ready entity=mon.1 client_cookie=d376de2cf410a968 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:31.443 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.439+0000 7fbf7959a700 1 -- 172.21.15.40:0/3044955964 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf7400b9a0 con 0x7fbf7c0b1b60 2020-11-20T13:56:31.443 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.439+0000 7fbf7959a700 1 -- 172.21.15.40:0/3044955964 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf7400bb00 con 0x7fbf7c0b1b60 2020-11-20T13:56:31.443 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.439+0000 7fbf7959a700 1 -- 172.21.15.40:0/3044955964 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf7400bc70 con 0x7fbf7c0b1b60 2020-11-20T13:56:31.443 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.439+0000 7fbf80b6f700 1 -- 172.21.15.40:0/3044955964 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf7c0b1b60 msgr2=0x7fbf7c144bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.443 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.439+0000 7fbf80b6f700 1 --2- 172.21.15.40:0/3044955964 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf7c0b1b60 0x7fbf7c144bd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fbf740032b0 tx=0x7fbf740091a0).stop 2020-11-20T13:56:31.444 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.440+0000 7fbf80b6f700 1 -- 172.21.15.40:0/3044955964 shutdown_connections 2020-11-20T13:56:31.444 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.440+0000 7fbf80b6f700 1 --2- 172.21.15.40:0/3044955964 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf7c0b1b60 0x7fbf7c144bd0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.444 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.440+0000 7fbf80b6f700 1 -- 172.21.15.40:0/3044955964 >> 172.21.15.40:0/3044955964 conn(0x7fbf7c0ba1d0 msgr2=0x7fbf7c0b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:31.445 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.440+0000 7fbf80b6f700 1 -- 172.21.15.40:0/3044955964 shutdown_connections 2020-11-20T13:56:31.445 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.440+0000 7fbf80b6f700 1 -- 172.21.15.40:0/3044955964 wait complete. 2020-11-20T13:56:31.448 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.441+0000 7fbf80b6f700 1 Processor -- start 2020-11-20T13:56:31.448 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.441+0000 7fbf80b6f700 1 -- start start 2020-11-20T13:56:31.448 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.441+0000 7fbf80b6f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf7c0b1b60 0x7fbf7c066c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.449 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.441+0000 7fbf80b6f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf7c0b31f0 0x7fbf7c12f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.449 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.441+0000 7fbf80b6f700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbf7c0b6c80 0x7fbf7c153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.449 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.441+0000 7fbf80b6f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c067110 con 0x7fbf7c0b1b60 2020-11-20T13:56:31.449 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.441+0000 7fbf80b6f700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c12cbd0 con 0x7fbf7c0b31f0 2020-11-20T13:56:31.449 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.441+0000 7fbf80b6f700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c130060 con 0x7fbf7c0b6c80 2020-11-20T13:56:31.450 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.442+0000 7fbf7a59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf7c0b1b60 0x7fbf7c066c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.450 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.442+0000 7fbf7a59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf7c0b1b60 0x7fbf7c066c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54770/0 (socket says 172.21.15.40:54770) 2020-11-20T13:56:31.450 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.442+0000 7fbf7a59c700 1 -- 172.21.15.40:0/3633230471 learned_addr learned my addr 172.21.15.40:0/3633230471 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:31.450 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.442+0000 7fbf7ad9d700 1 --2- 172.21.15.40:0/3633230471 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbf7c0b6c80 0x7fbf7c153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.450 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.442+0000 7fbf79d9b700 1 --2- 172.21.15.40:0/3633230471 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf7c0b31f0 0x7fbf7c12f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.451 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.442+0000 7fbf7a59c700 1 -- 172.21.15.40:0/3633230471 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbf7c0b6c80 msgr2=0x7fbf7c153f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.451 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.442+0000 7fbf7a59c700 1 --2- 172.21.15.40:0/3633230471 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbf7c0b6c80 0x7fbf7c153f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.451 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.442+0000 7fbf7a59c700 1 -- 172.21.15.40:0/3633230471 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf7c0b31f0 msgr2=0x7fbf7c12f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.451 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.442+0000 7fbf7a59c700 1 --2- 172.21.15.40:0/3633230471 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf7c0b31f0 0x7fbf7c12f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.451 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.442+0000 7fbf7a59c700 1 -- 172.21.15.40:0/3633230471 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf7c130300 con 0x7fbf7c0b1b60 2020-11-20T13:56:31.452 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.442+0000 7fbf7a59c700 1 --2- 172.21.15.40:0/3633230471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf7c0b1b60 0x7fbf7c066c20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fbf70001250 tx=0x7fbf700027a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:31.452 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.442+0000 7fbf6b7fe700 1 -- 172.21.15.40:0/3633230471 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf700072e0 con 0x7fbf7c0b1b60 2020-11-20T13:56:31.452 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.442+0000 7fbf80b6f700 1 -- 172.21.15.40:0/3633230471 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf7c154b60 con 0x7fbf7c0b1b60 2020-11-20T13:56:31.452 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.442+0000 7fbf6b7fe700 1 -- 172.21.15.40:0/3633230471 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf700054b0 con 0x7fbf7c0b1b60 2020-11-20T13:56:31.452 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.442+0000 7fbf80b6f700 1 -- 172.21.15.40:0/3633230471 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf7c063c80 con 0x7fbf7c0b1b60 2020-11-20T13:56:31.453 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.442+0000 7fbf6b7fe700 1 -- 172.21.15.40:0/3633230471 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf70005810 con 0x7fbf7c0b1b60 2020-11-20T13:56:31.453 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.443+0000 7fbf80b6f700 1 -- 172.21.15.40:0/3633230471 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbf7c130490 con 0x7fbf7c0b1b60 2020-11-20T13:56:31.453 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.445+0000 7fbf6b7fe700 1 -- 172.21.15.40:0/3633230471 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fbf70010070 con 0x7fbf7c0b1b60 2020-11-20T13:56:31.453 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.445+0000 7fbf6b7fe700 1 --2- 172.21.15.40:0/3633230471 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fbf580513c0 0x7fbf58053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.454 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.445+0000 7fbf6b7fe700 1 -- 172.21.15.40:0/3633230471 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fbf700690a0 con 0x7fbf7c0b1b60 2020-11-20T13:56:31.454 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.445+0000 7fbf6b7fe700 1 --2- 172.21.15.40:0/3633230471 >> [v2:172.21.15.116:6802/23171,v1:172.21.15.116:6803/23171] conn(0x7fbf58058e50 0x7fbf5805b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.454 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.445+0000 7fbf6b7fe700 1 -- 172.21.15.40:0/3633230471 --> [v2:172.21.15.116:6802/23171,v1:172.21.15.116:6803/23171] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbf5805b8c0 con 0x7fbf58058e50 2020-11-20T13:56:31.454 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.445+0000 7fbf6b7fe700 1 -- 172.21.15.40:0/3633230471 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf70069460 con 0x7fbf7c0b1b60 2020-11-20T13:56:31.454 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.445+0000 7fbf79d9b700 1 --2- 172.21.15.40:0/3633230471 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fbf580513c0 0x7fbf58053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.455 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.446+0000 7fbf7ad9d700 1 --2- 172.21.15.40:0/3633230471 >> [v2:172.21.15.116:6802/23171,v1:172.21.15.116:6803/23171] conn(0x7fbf58058e50 0x7fbf5805b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.456 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.446+0000 7fbf79d9b700 1 --2- 172.21.15.40:0/3633230471 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fbf580513c0 0x7fbf58053880 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fbf6c00d000 tx=0x7fbf6c0078a0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:31.456 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.446+0000 7fbf7ad9d700 1 --2- 172.21.15.40:0/3633230471 >> [v2:172.21.15.116:6802/23171,v1:172.21.15.116:6803/23171] conn(0x7fbf58058e50 0x7fbf5805b270 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:31.456 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.447+0000 7fbf6b7fe700 1 -- 172.21.15.40:0/3633230471 <== osd.5 v2:172.21.15.116:6802/23171 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7fbf5805b8c0 con 0x7fbf58058e50 2020-11-20T13:56:31.465 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.465+0000 7fbf80b6f700 1 -- 172.21.15.40:0/3633230471 --> [v2:172.21.15.116:6802/23171,v1:172.21.15.116:6803/23171] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbf7c057050 con 0x7fbf58058e50 2020-11-20T13:56:31.466 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.465+0000 7fbf6b7fe700 1 -- 172.21.15.40:0/3633230471 <== osd.5 v2:172.21.15.116:6802/23171 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbf7c057050 con 0x7fbf58058e50 2020-11-20T13:56:31.466 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.466+0000 7fbf80b6f700 1 -- 172.21.15.40:0/3633230471 >> [v2:172.21.15.116:6802/23171,v1:172.21.15.116:6803/23171] conn(0x7fbf58058e50 msgr2=0x7fbf5805b270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.467 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.466+0000 7fbf80b6f700 1 --2- 172.21.15.40:0/3633230471 >> [v2:172.21.15.116:6802/23171,v1:172.21.15.116:6803/23171] conn(0x7fbf58058e50 0x7fbf5805b270 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.467 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.466+0000 7fbf80b6f700 1 -- 172.21.15.40:0/3633230471 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fbf580513c0 msgr2=0x7fbf58053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.467 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.466+0000 7fbf80b6f700 1 --2- 172.21.15.40:0/3633230471 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fbf580513c0 0x7fbf58053880 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fbf6c00d000 tx=0x7fbf6c0078a0).stop 2020-11-20T13:56:31.467 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.466+0000 7fbf80b6f700 1 -- 172.21.15.40:0/3633230471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf7c0b1b60 msgr2=0x7fbf7c066c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.468 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.466+0000 7fbf80b6f700 1 --2- 172.21.15.40:0/3633230471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf7c0b1b60 0x7fbf7c066c20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fbf70001250 tx=0x7fbf700027a0).stop 2020-11-20T13:56:31.468 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.466+0000 7fbf7ad9d700 1 -- 172.21.15.40:0/3633230471 reap_dead start 2020-11-20T13:56:31.468 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.466+0000 7fbf80b6f700 1 -- 172.21.15.40:0/3633230471 shutdown_connections 2020-11-20T13:56:31.468 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.466+0000 7fbf80b6f700 1 -- 172.21.15.40:0/3633230471 >> 172.21.15.40:0/3633230471 conn(0x7fbf7c0ba1d0 msgr2=0x7fbf7c05fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:31.468 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.466+0000 7fbf80b6f700 1 -- 172.21.15.40:0/3633230471 shutdown_connections 2020-11-20T13:56:31.469 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.466+0000 7fbf80b6f700 1 -- 172.21.15.40:0/3633230471 wait complete. 2020-11-20T13:56:31.477 INFO:teuthology.orchestra.run.smithi040.stdout:55834574854 2020-11-20T13:56:31.477 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.6 flush_pg_stats 2020-11-20T13:56:31.561 INFO:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T13:56:31.563 INFO:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T13:56:31.593 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.593+0000 7f26ca1a1700 1 Processor -- start 2020-11-20T13:56:31.594 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.593+0000 7f26ca1a1700 1 -- start start 2020-11-20T13:56:31.596 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.594+0000 7f26ca1a1700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f26c4147130 0x7f26c40b7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.596 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.594+0000 7f26ca1a1700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f26c40647c0 0x7f26c40b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.596 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.594+0000 7f26ca1a1700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26c414ab10 con 0x7f26c40b1480 2020-11-20T13:56:31.597 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.594+0000 7f26ca1a1700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26c414adb0 con 0x7f26c4147130 2020-11-20T13:56:31.597 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.594+0000 7f26ca1a1700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26c414b050 con 0x7f26c40647c0 2020-11-20T13:56:31.597 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.594+0000 7f26c2ffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f26c4147130 0x7f26c40b7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.597 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.594+0000 7f26c2ffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f26c4147130 0x7f26c40b7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54782/0 (socket says 172.21.15.40:54782) 2020-11-20T13:56:31.597 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.594+0000 7f26c2ffd700 1 -- 172.21.15.40:0/2943406282 learned_addr learned my addr 172.21.15.40:0/2943406282 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:31.598 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.594+0000 7f26c3fff700 1 --2- 172.21.15.40:0/2943406282 >> v2:172.21.15.116:3300/0 conn(0x7f26c40647c0 0x7f26c40b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.598 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.594+0000 7f26c2ffd700 1 -- 172.21.15.40:0/2943406282 >> v2:172.21.15.116:3300/0 conn(0x7f26c40647c0 msgr2=0x7f26c40b7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:31.598 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.594+0000 7f26c2ffd700 1 --2- 172.21.15.40:0/2943406282 >> v2:172.21.15.116:3300/0 conn(0x7f26c40647c0 0x7f26c40b7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.598 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.594+0000 7f26c2ffd700 1 -- 172.21.15.40:0/2943406282 >> v1:172.21.15.116:6789/0 conn(0x7f26c40b1480 legacy=0x7f26c4146a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.599 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.594+0000 7f26c2ffd700 1 -- 172.21.15.40:0/2943406282 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c414b2f0 con 0x7f26c4147130 2020-11-20T13:56:31.599 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.594+0000 7f26c3fff700 1 --2- 172.21.15.40:0/2943406282 >> v2:172.21.15.116:3300/0 conn(0x7f26c40647c0 0x7f26c40b7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:31.599 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.594+0000 7f26c2ffd700 1 --2- 172.21.15.40:0/2943406282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26c4147130 0x7f26c40b7520 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f26ac002f30 tx=0x7f26ac006e80).ready entity=mon.0 client_cookie=566e24ad589cc1e8 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:31.599 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.595+0000 7f26c27fc700 1 -- 172.21.15.40:0/2943406282 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f26ac0097f0 con 0x7f26c4147130 2020-11-20T13:56:31.599 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.595+0000 7f26c27fc700 1 -- 172.21.15.40:0/2943406282 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26ac009950 con 0x7f26c4147130 2020-11-20T13:56:31.600 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.595+0000 7f26c27fc700 1 -- 172.21.15.40:0/2943406282 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f26ac009cb0 con 0x7f26c4147130 2020-11-20T13:56:31.600 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.595+0000 7f26ca1a1700 1 -- 172.21.15.40:0/2943406282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26c4147130 msgr2=0x7f26c40b7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.600 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.595+0000 7f26ca1a1700 1 --2- 172.21.15.40:0/2943406282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26c4147130 0x7f26c40b7520 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f26ac002f30 tx=0x7f26ac006e80).stop 2020-11-20T13:56:31.600 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.595+0000 7f26ca1a1700 1 -- 172.21.15.40:0/2943406282 shutdown_connections 2020-11-20T13:56:31.600 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.595+0000 7f26ca1a1700 1 --2- 172.21.15.40:0/2943406282 >> v2:172.21.15.116:3300/0 conn(0x7f26c40647c0 0x7f26c40b7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.600 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.595+0000 7f26ca1a1700 1 --2- 172.21.15.40:0/2943406282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26c4147130 0x7f26c40b7520 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.601 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.595+0000 7f26ca1a1700 1 -- 172.21.15.40:0/2943406282 >> 172.21.15.40:0/2943406282 conn(0x7f26c40bb330 msgr2=0x7f26c4063a60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:31.601 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.595+0000 7f26ca1a1700 1 -- 172.21.15.40:0/2943406282 shutdown_connections 2020-11-20T13:56:31.601 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.595+0000 7f26ca1a1700 1 -- 172.21.15.40:0/2943406282 wait complete. 2020-11-20T13:56:31.601 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.596+0000 7f26ca1a1700 1 Processor -- start 2020-11-20T13:56:31.604 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.596+0000 7f26ca1a1700 1 -- start start 2020-11-20T13:56:31.604 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.597+0000 7f26ca1a1700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f26c40647c0 0x7f26c4141040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.604 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.597+0000 7f26ca1a1700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26c40b1480 0x7f26c41498b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.605 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.597+0000 7f26ca1a1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26c4147130 0x7f26c4138110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.605 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.597+0000 7f26ca1a1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c414a000 con 0x7f26c4147130 2020-11-20T13:56:31.605 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.597+0000 7f26ca1a1700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c4149da0 con 0x7f26c40b1480 2020-11-20T13:56:31.605 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.597+0000 7f26ca1a1700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26c4142460 con 0x7f26c40647c0 2020-11-20T13:56:31.605 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.597+0000 7f26c3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26c4147130 0x7f26c4138110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.606 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.597+0000 7f26c3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26c4147130 0x7f26c4138110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54790/0 (socket says 172.21.15.40:54790) 2020-11-20T13:56:31.606 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.597+0000 7f26c3fff700 1 -- 172.21.15.40:0/3799663216 learned_addr learned my addr 172.21.15.40:0/3799663216 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:31.606 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.597+0000 7f26c2ffd700 1 --2- 172.21.15.40:0/3799663216 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26c40b1480 0x7f26c41498b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.606 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.597+0000 7f26c37fe700 1 --2- 172.21.15.40:0/3799663216 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f26c40647c0 0x7f26c4141040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.606 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.597+0000 7f26c3fff700 1 -- 172.21.15.40:0/3799663216 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f26c40647c0 msgr2=0x7f26c4141040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.607 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.597+0000 7f26c3fff700 1 --2- 172.21.15.40:0/3799663216 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f26c40647c0 0x7f26c4141040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.607 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.597+0000 7f26c3fff700 1 -- 172.21.15.40:0/3799663216 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26c40b1480 msgr2=0x7f26c41498b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.607 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.597+0000 7f26c3fff700 1 --2- 172.21.15.40:0/3799663216 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26c40b1480 0x7f26c41498b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.607 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.597+0000 7f26c3fff700 1 -- 172.21.15.40:0/3799663216 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c4142700 con 0x7f26c4147130 2020-11-20T13:56:31.607 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.598+0000 7f26c3fff700 1 --2- 172.21.15.40:0/3799663216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26c4147130 0x7f26c4138110 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f26bc006dd0 tx=0x7f26bc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:31.608 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.598+0000 7f26c0ff9700 1 -- 172.21.15.40:0/3799663216 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f26bc00c810 con 0x7f26c4147130 2020-11-20T13:56:31.608 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.598+0000 7f26c0ff9700 1 -- 172.21.15.40:0/3799663216 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26bc00c970 con 0x7f26c4147130 2020-11-20T13:56:31.608 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.598+0000 7f26ca1a1700 1 -- 172.21.15.40:0/3799663216 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26c4147a50 con 0x7f26c4147130 2020-11-20T13:56:31.608 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.598+0000 7f26c0ff9700 1 -- 172.21.15.40:0/3799663216 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f26bc00ccd0 con 0x7f26c4147130 2020-11-20T13:56:31.608 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.598+0000 7f26ca1a1700 1 -- 172.21.15.40:0/3799663216 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26c4138600 con 0x7f26c4147130 2020-11-20T13:56:31.609 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.599+0000 7f26ca1a1700 1 -- 172.21.15.40:0/3799663216 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f26c4066cd0 con 0x7f26c4147130 2020-11-20T13:56:31.609 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.599+0000 7f26c0ff9700 1 -- 172.21.15.40:0/3799663216 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f26bc01d460 con 0x7f26c4147130 2020-11-20T13:56:31.609 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.599+0000 7f26c0ff9700 1 --2- 172.21.15.40:0/3799663216 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f26a8051320 0x7f26a80537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.609 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.599+0000 7f26c0ff9700 1 -- 172.21.15.40:0/3799663216 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f26bc0694e0 con 0x7f26c4147130 2020-11-20T13:56:31.609 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.599+0000 7f26c37fe700 1 --2- 172.21.15.40:0/3799663216 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f26a8051320 0x7f26a80537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.610 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.599+0000 7f26c0ff9700 1 --2- 172.21.15.40:0/3799663216 >> [v2:172.21.15.116:6810/23172,v1:172.21.15.116:6811/23172] conn(0x7f26a8058e30 0x7f26a805b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.610 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.599+0000 7f26c0ff9700 1 -- 172.21.15.40:0/3799663216 --> [v2:172.21.15.116:6810/23172,v1:172.21.15.116:6811/23172] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f26a805b8a0 con 0x7f26a8058e30 2020-11-20T13:56:31.610 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.599+0000 7f26c0ff9700 1 -- 172.21.15.40:0/3799663216 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26bc06bb70 con 0x7f26c4147130 2020-11-20T13:56:31.610 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.600+0000 7f26c2ffd700 1 --2- 172.21.15.40:0/3799663216 >> [v2:172.21.15.116:6810/23172,v1:172.21.15.116:6811/23172] conn(0x7f26a8058e30 0x7f26a805b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.610 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.600+0000 7f26c37fe700 1 --2- 172.21.15.40:0/3799663216 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f26a8051320 0x7f26a80537e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f26b400d000 tx=0x7f26b4007ad0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:31.611 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.600+0000 7f26c2ffd700 1 --2- 172.21.15.40:0/3799663216 >> [v2:172.21.15.116:6810/23172,v1:172.21.15.116:6811/23172] conn(0x7f26a8058e30 0x7f26a805b250 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:31.611 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.601+0000 7f26c0ff9700 1 -- 172.21.15.40:0/3799663216 <== osd.6 v2:172.21.15.116:6810/23172 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7f26a805b8a0 con 0x7f26a8058e30 2020-11-20T13:56:31.619 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.619+0000 7f26ca1a1700 1 -- 172.21.15.40:0/3799663216 --> [v2:172.21.15.116:6810/23172,v1:172.21.15.116:6811/23172] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f26c40b77a0 con 0x7f26a8058e30 2020-11-20T13:56:31.620 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.620+0000 7f26c0ff9700 1 -- 172.21.15.40:0/3799663216 <== osd.6 v2:172.21.15.116:6810/23172 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f26c40b77a0 con 0x7f26a8058e30 2020-11-20T13:56:31.621 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.620+0000 7f26ca1a1700 1 -- 172.21.15.40:0/3799663216 >> [v2:172.21.15.116:6810/23172,v1:172.21.15.116:6811/23172] conn(0x7f26a8058e30 msgr2=0x7f26a805b250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.622 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.620+0000 7f26ca1a1700 1 --2- 172.21.15.40:0/3799663216 >> [v2:172.21.15.116:6810/23172,v1:172.21.15.116:6811/23172] conn(0x7f26a8058e30 0x7f26a805b250 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.622 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.620+0000 7f26ca1a1700 1 -- 172.21.15.40:0/3799663216 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f26a8051320 msgr2=0x7f26a80537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.623 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.620+0000 7f26ca1a1700 1 --2- 172.21.15.40:0/3799663216 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f26a8051320 0x7f26a80537e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f26b400d000 tx=0x7f26b4007ad0).stop 2020-11-20T13:56:31.623 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.620+0000 7f26ca1a1700 1 -- 172.21.15.40:0/3799663216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26c4147130 msgr2=0x7f26c4138110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.623 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.620+0000 7f26ca1a1700 1 --2- 172.21.15.40:0/3799663216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26c4147130 0x7f26c4138110 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f26bc006dd0 tx=0x7f26bc008040).stop 2020-11-20T13:56:31.624 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.620+0000 7f26c3fff700 1 -- 172.21.15.40:0/3799663216 reap_dead start 2020-11-20T13:56:31.624 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.620+0000 7f26ca1a1700 1 -- 172.21.15.40:0/3799663216 shutdown_connections 2020-11-20T13:56:31.624 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.620+0000 7f26ca1a1700 1 -- 172.21.15.40:0/3799663216 >> 172.21.15.40:0/3799663216 conn(0x7f26c40bb330 msgr2=0x7f26c41458d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:31.624 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.620+0000 7f26ca1a1700 1 -- 172.21.15.40:0/3799663216 shutdown_connections 2020-11-20T13:56:31.625 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.620+0000 7f26ca1a1700 1 -- 172.21.15.40:0/3799663216 wait complete. 2020-11-20T13:56:31.630 INFO:teuthology.orchestra.run.smithi040.stdout:55834574853 2020-11-20T13:56:31.630 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.7 flush_pg_stats 2020-11-20T13:56:31.787 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.787+0000 7fcefffff700 1 Processor -- start 2020-11-20T13:56:31.787 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.787+0000 7fcefffff700 1 -- start start 2020-11-20T13:56:31.789 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.788+0000 7fcefffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcf000629f0 0x7fcf00062dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.790 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.788+0000 7fcefffff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fcf000b2630 0x7fcf000b2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.790 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.788+0000 7fcefffff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf00063370 con 0x7fcf000aff30 2020-11-20T13:56:31.790 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.788+0000 7fcefffff700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf000b3fd0 con 0x7fcf000629f0 2020-11-20T13:56:31.790 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.788+0000 7fcefffff700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf00144290 con 0x7fcf000b2630 2020-11-20T13:56:31.790 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.788+0000 7fcefeffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcf000629f0 0x7fcf00062dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.791 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.788+0000 7fcefe7fc700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fcf000aff30 0x7fcf00142e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45396/0 (socket says 172.21.15.40:45396) 2020-11-20T13:56:31.791 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.788+0000 7fcefe7fc700 1 -- 172.21.15.40:0/229557933 learned_addr learned my addr 172.21.15.40:0/229557933 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:31.791 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.788+0000 7fceff7fe700 1 --2- 172.21.15.40:0/229557933 >> v2:172.21.15.116:3300/0 conn(0x7fcf000b2630 0x7fcf000b2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.791 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.788+0000 7fcefeffd700 1 -- 172.21.15.40:0/229557933 >> v2:172.21.15.116:3300/0 conn(0x7fcf000b2630 msgr2=0x7fcf000b2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:31.791 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.788+0000 7fcefeffd700 1 --2- 172.21.15.40:0/229557933 >> v2:172.21.15.116:3300/0 conn(0x7fcf000b2630 0x7fcf000b2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.792 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.788+0000 7fcefeffd700 1 -- 172.21.15.40:0/229557933 >> v1:172.21.15.40:6789/0 conn(0x7fcf000aff30 legacy=0x7fcf00142e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.792 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.788+0000 7fcefdffb700 1 -- 172.21.15.40:0/229557933 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2289587145 0 0) 0x7fcf00063370 con 0x7fcf000aff30 2020-11-20T13:56:31.792 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.788+0000 7fcefeffd700 1 -- 172.21.15.40:0/229557933 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf00144530 con 0x7fcf000629f0 2020-11-20T13:56:31.793 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.788+0000 7fceff7fe700 1 --2- 172.21.15.40:0/229557933 >> v2:172.21.15.116:3300/0 conn(0x7fcf000b2630 0x7fcf000b2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:31.793 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.788+0000 7fcefeffd700 1 --2- 172.21.15.40:0/229557933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf000629f0 0x7fcf00062dd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fcef4002f30 tx=0x7fcef4006dc0).ready entity=mon.0 client_cookie=3b316beee685401a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:31.793 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.788+0000 7fcefdffb700 1 -- 172.21.15.40:0/229557933 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcef40097f0 con 0x7fcf000629f0 2020-11-20T13:56:31.793 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.788+0000 7fcefdffb700 1 -- 172.21.15.40:0/229557933 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef4009950 con 0x7fcf000629f0 2020-11-20T13:56:31.793 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.788+0000 7fcefdffb700 1 -- 172.21.15.40:0/229557933 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcef4009cb0 con 0x7fcf000629f0 2020-11-20T13:56:31.794 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.788+0000 7fcefffff700 1 -- 172.21.15.40:0/229557933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf000629f0 msgr2=0x7fcf00062dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.794 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.788+0000 7fcefffff700 1 --2- 172.21.15.40:0/229557933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf000629f0 0x7fcf00062dd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fcef4002f30 tx=0x7fcef4006dc0).stop 2020-11-20T13:56:31.794 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.789+0000 7fcefffff700 1 -- 172.21.15.40:0/229557933 shutdown_connections 2020-11-20T13:56:31.794 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.789+0000 7fcefffff700 1 --2- 172.21.15.40:0/229557933 >> v2:172.21.15.116:3300/0 conn(0x7fcf000b2630 0x7fcf000b2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.794 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.789+0000 7fcefffff700 1 --2- 172.21.15.40:0/229557933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf000629f0 0x7fcf00062dd0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.795 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.789+0000 7fcefffff700 1 -- 172.21.15.40:0/229557933 >> 172.21.15.40:0/229557933 conn(0x7fcf0005f390 msgr2=0x7fcf0005f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:31.795 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.789+0000 7fcefffff700 1 -- 172.21.15.40:0/229557933 shutdown_connections 2020-11-20T13:56:31.795 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.789+0000 7fcefffff700 1 -- 172.21.15.40:0/229557933 wait complete. 2020-11-20T13:56:31.799 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.790+0000 7fcefffff700 1 Processor -- start 2020-11-20T13:56:31.799 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.790+0000 7fcefffff700 1 -- start start 2020-11-20T13:56:31.799 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.791+0000 7fcefffff700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcf000629f0 0x7fcf00157f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.800 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.791+0000 7fcefffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf000aff30 0x7fcf000af280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.800 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.791+0000 7fcefffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf000b2630 0x7fcf00142210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.800 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.791+0000 7fcefffff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf001575a0 con 0x7fcf000aff30 2020-11-20T13:56:31.800 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.791+0000 7fcefffff700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf00157840 con 0x7fcf000b2630 2020-11-20T13:56:31.800 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.791+0000 7fcefffff700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf00157ae0 con 0x7fcf000629f0 2020-11-20T13:56:31.801 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.791+0000 7fcefe7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf000aff30 0x7fcf000af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.801 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.791+0000 7fcefe7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf000aff30 0x7fcf000af280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54804/0 (socket says 172.21.15.40:54804) 2020-11-20T13:56:31.801 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.791+0000 7fcefe7fc700 1 -- 172.21.15.40:0/892666455 learned_addr learned my addr 172.21.15.40:0/892666455 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:31.801 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.791+0000 7fcefeffd700 1 --2- 172.21.15.40:0/892666455 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcf000629f0 0x7fcf00157f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.802 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.791+0000 7fceff7fe700 1 --2- 172.21.15.40:0/892666455 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf000b2630 0x7fcf00142210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.802 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.791+0000 7fcefe7fc700 1 -- 172.21.15.40:0/892666455 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcf000629f0 msgr2=0x7fcf00157f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.802 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.791+0000 7fcefe7fc700 1 --2- 172.21.15.40:0/892666455 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcf000629f0 0x7fcf00157f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.802 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.791+0000 7fcefe7fc700 1 -- 172.21.15.40:0/892666455 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf000b2630 msgr2=0x7fcf00142210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.802 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.791+0000 7fcefe7fc700 1 --2- 172.21.15.40:0/892666455 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf000b2630 0x7fcf00142210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.803 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.791+0000 7fcefe7fc700 1 -- 172.21.15.40:0/892666455 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf00066be0 con 0x7fcf000aff30 2020-11-20T13:56:31.803 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.791+0000 7fceff7fe700 1 --2- 172.21.15.40:0/892666455 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf000b2630 0x7fcf00142210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:31.803 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.791+0000 7fcefe7fc700 1 --2- 172.21.15.40:0/892666455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf000aff30 0x7fcf000af280 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fcef0000e90 tx=0x7fcef0007a90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:31.805 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.791+0000 7fceeffff700 1 -- 172.21.15.40:0/892666455 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcef00072e0 con 0x7fcf000aff30 2020-11-20T13:56:31.805 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.791+0000 7fceeffff700 1 -- 172.21.15.40:0/892666455 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef0007440 con 0x7fcf000aff30 2020-11-20T13:56:31.806 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.791+0000 7fceeffff700 1 -- 172.21.15.40:0/892666455 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcef00057d0 con 0x7fcf000aff30 2020-11-20T13:56:31.806 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.791+0000 7fcefffff700 1 -- 172.21.15.40:0/892666455 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf00142700 con 0x7fcf000aff30 2020-11-20T13:56:31.806 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.791+0000 7fcefffff700 1 -- 172.21.15.40:0/892666455 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf0013b1c0 con 0x7fcf000aff30 2020-11-20T13:56:31.806 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.792+0000 7fcefffff700 1 -- 172.21.15.40:0/892666455 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcf00157420 con 0x7fcf000aff30 2020-11-20T13:56:31.806 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.792+0000 7fceeffff700 1 -- 172.21.15.40:0/892666455 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fcef0010070 con 0x7fcf000aff30 2020-11-20T13:56:31.807 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.793+0000 7fceeffff700 1 --2- 172.21.15.40:0/892666455 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fcedc0513c0 0x7fcedc053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.807 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.793+0000 7fceeffff700 1 -- 172.21.15.40:0/892666455 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fcef0068820 con 0x7fcf000aff30 2020-11-20T13:56:31.807 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.793+0000 7fceeffff700 1 --2- 172.21.15.40:0/892666455 >> [v2:172.21.15.116:6826/23196,v1:172.21.15.116:6827/23196] conn(0x7fcedc058ed0 0x7fcedc05b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.807 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.793+0000 7fcefeffd700 1 --2- 172.21.15.40:0/892666455 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fcedc0513c0 0x7fcedc053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.807 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.793+0000 7fceeffff700 1 -- 172.21.15.40:0/892666455 --> [v2:172.21.15.116:6826/23196,v1:172.21.15.116:6827/23196] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcedc05b940 con 0x7fcedc058ed0 2020-11-20T13:56:31.808 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.793+0000 7fceeffff700 1 -- 172.21.15.40:0/892666455 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcef006d240 con 0x7fcf000aff30 2020-11-20T13:56:31.808 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.793+0000 7fceff7fe700 1 --2- 172.21.15.40:0/892666455 >> [v2:172.21.15.116:6826/23196,v1:172.21.15.116:6827/23196] conn(0x7fcedc058ed0 0x7fcedc05b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.808 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.793+0000 7fcefeffd700 1 --2- 172.21.15.40:0/892666455 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fcedc0513c0 0x7fcedc053880 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fcef401b000 tx=0x7fcef4015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:31.808 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.793+0000 7fceff7fe700 1 --2- 172.21.15.40:0/892666455 >> [v2:172.21.15.116:6826/23196,v1:172.21.15.116:6827/23196] conn(0x7fcedc058ed0 0x7fcedc05b2f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:31.808 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.794+0000 7fceeffff700 1 -- 172.21.15.40:0/892666455 <== osd.7 v2:172.21.15.116:6826/23196 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7fcedc05b940 con 0x7fcedc058ed0 2020-11-20T13:56:31.814 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.812+0000 7fcefffff700 1 -- 172.21.15.40:0/892666455 --> [v2:172.21.15.116:6826/23196,v1:172.21.15.116:6827/23196] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcf000b6330 con 0x7fcedc058ed0 2020-11-20T13:56:31.815 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.813+0000 7fceeffff700 1 -- 172.21.15.40:0/892666455 <== osd.7 v2:172.21.15.116:6826/23196 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcf000b6330 con 0x7fcedc058ed0 2020-11-20T13:56:31.815 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.813+0000 7fcefffff700 1 -- 172.21.15.40:0/892666455 >> [v2:172.21.15.116:6826/23196,v1:172.21.15.116:6827/23196] conn(0x7fcedc058ed0 msgr2=0x7fcedc05b2f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.815 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.813+0000 7fcefffff700 1 --2- 172.21.15.40:0/892666455 >> [v2:172.21.15.116:6826/23196,v1:172.21.15.116:6827/23196] conn(0x7fcedc058ed0 0x7fcedc05b2f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.815 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.813+0000 7fcefffff700 1 -- 172.21.15.40:0/892666455 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fcedc0513c0 msgr2=0x7fcedc053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.815 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.813+0000 7fcefffff700 1 --2- 172.21.15.40:0/892666455 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fcedc0513c0 0x7fcedc053880 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fcef401b000 tx=0x7fcef4015000).stop 2020-11-20T13:56:31.816 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.813+0000 7fcefffff700 1 -- 172.21.15.40:0/892666455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf000aff30 msgr2=0x7fcf000af280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.816 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.813+0000 7fcefffff700 1 --2- 172.21.15.40:0/892666455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf000aff30 0x7fcf000af280 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fcef0000e90 tx=0x7fcef0007a90).stop 2020-11-20T13:56:31.816 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.813+0000 7fceff7fe700 1 -- 172.21.15.40:0/892666455 reap_dead start 2020-11-20T13:56:31.816 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.813+0000 7fcefffff700 1 -- 172.21.15.40:0/892666455 shutdown_connections 2020-11-20T13:56:31.816 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.813+0000 7fcefffff700 1 -- 172.21.15.40:0/892666455 >> 172.21.15.40:0/892666455 conn(0x7fcf0005f390 msgr2=0x7fcf0013be70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:31.817 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.813+0000 7fcefffff700 1 -- 172.21.15.40:0/892666455 shutdown_connections 2020-11-20T13:56:31.817 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.813+0000 7fcefffff700 1 -- 172.21.15.40:0/892666455 wait complete. 2020-11-20T13:56:31.823 INFO:teuthology.orchestra.run.smithi040.stdout:55834574854 2020-11-20T13:56:31.824 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.0 2020-11-20T13:56:31.940 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.939+0000 7f1f37930700 1 Processor -- start 2020-11-20T13:56:31.940 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.940+0000 7f1f37930700 1 -- start start 2020-11-20T13:56:31.942 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.940+0000 7f1f37930700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1f300b5bf0 0x7f1f3005f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.943 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.940+0000 7f1f37930700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1f300b3960 0x7f1f3014cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.943 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.940+0000 7f1f37930700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f300b5fd0 con 0x7f1f30144390 2020-11-20T13:56:31.943 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.940+0000 7f1f37930700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f300b05f0 con 0x7f1f300b3960 2020-11-20T13:56:31.943 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.940+0000 7f1f37930700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f30144870 con 0x7f1f300b5bf0 2020-11-20T13:56:31.943 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.940+0000 7f1f35ecd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1f300b3960 0x7f1f3014cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.944 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.940+0000 7f1f34ecb700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1f30144390 0x7f1f3005f880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45408/0 (socket says 172.21.15.40:45408) 2020-11-20T13:56:31.944 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.940+0000 7f1f34ecb700 1 -- 172.21.15.40:0/2564937694 learned_addr learned my addr 172.21.15.40:0/2564937694 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:31.944 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.941+0000 7f1f356cc700 1 --2- 172.21.15.40:0/2564937694 >> v2:172.21.15.116:3300/0 conn(0x7f1f300b5bf0 0x7f1f3005f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.944 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.941+0000 7f1f35ecd700 1 -- 172.21.15.40:0/2564937694 >> v2:172.21.15.116:3300/0 conn(0x7f1f300b5bf0 msgr2=0x7f1f3005f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:31.944 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.941+0000 7f1f35ecd700 1 --2- 172.21.15.40:0/2564937694 >> v2:172.21.15.116:3300/0 conn(0x7f1f300b5bf0 0x7f1f3005f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.945 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.941+0000 7f1f35ecd700 1 -- 172.21.15.40:0/2564937694 >> v1:172.21.15.40:6789/0 conn(0x7f1f30144390 legacy=0x7f1f3005f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.945 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.941+0000 7f1f35ecd700 1 -- 172.21.15.40:0/2564937694 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f300b0c30 con 0x7f1f300b3960 2020-11-20T13:56:31.945 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.941+0000 7f1f23fff700 1 -- 172.21.15.40:0/2564937694 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 880553134 0 0) 0x7f1f300b5fd0 con 0x7f1f30144390 2020-11-20T13:56:31.945 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.941+0000 7f1f35ecd700 1 --2- 172.21.15.40:0/2564937694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f300b3960 0x7f1f3014cf50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f1f280032b0 tx=0x7f1f280091a0).ready entity=mon.0 client_cookie=decdbe79aff4f24c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:31.945 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.941+0000 7f1f23fff700 1 -- 172.21.15.40:0/2564937694 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f2800b9a0 con 0x7f1f300b3960 2020-11-20T13:56:31.946 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.941+0000 7f1f23fff700 1 -- 172.21.15.40:0/2564937694 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f2800bb00 con 0x7f1f300b3960 2020-11-20T13:56:31.946 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.941+0000 7f1f23fff700 1 -- 172.21.15.40:0/2564937694 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f2800bc70 con 0x7f1f300b3960 2020-11-20T13:56:31.946 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.941+0000 7f1f37930700 1 -- 172.21.15.40:0/2564937694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f300b3960 msgr2=0x7f1f3014cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.946 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.941+0000 7f1f37930700 1 --2- 172.21.15.40:0/2564937694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f300b3960 0x7f1f3014cf50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f1f280032b0 tx=0x7f1f280091a0).stop 2020-11-20T13:56:31.946 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.942+0000 7f1f37930700 1 -- 172.21.15.40:0/2564937694 shutdown_connections 2020-11-20T13:56:31.947 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.942+0000 7f1f37930700 1 --2- 172.21.15.40:0/2564937694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f300b3960 0x7f1f3014cf50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.947 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.942+0000 7f1f37930700 1 --2- 172.21.15.40:0/2564937694 >> v2:172.21.15.116:3300/0 conn(0x7f1f300b5bf0 0x7f1f3005f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.947 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.942+0000 7f1f37930700 1 -- 172.21.15.40:0/2564937694 >> 172.21.15.40:0/2564937694 conn(0x7f1f300ba1d0 msgr2=0x7f1f300b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:31.947 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.942+0000 7f1f37930700 1 -- 172.21.15.40:0/2564937694 shutdown_connections 2020-11-20T13:56:31.947 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.942+0000 7f1f37930700 1 -- 172.21.15.40:0/2564937694 wait complete. 2020-11-20T13:56:31.948 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.943+0000 7f1f37930700 1 Processor -- start 2020-11-20T13:56:31.948 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.943+0000 7f1f37930700 1 -- start start 2020-11-20T13:56:31.950 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.943+0000 7f1f37930700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1f300b3960 0x7f1f30140c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.951 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.943+0000 7f1f37930700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f300b5bf0 0x7f1f30067000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.951 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.943+0000 7f1f37930700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f30144390 0x7f1f30135dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.951 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.943+0000 7f1f37930700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f300adb60 con 0x7f1f300b5bf0 2020-11-20T13:56:31.951 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.943+0000 7f1f37930700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f300af940 con 0x7f1f30144390 2020-11-20T13:56:31.951 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.943+0000 7f1f37930700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f30140690 con 0x7f1f300b3960 2020-11-20T13:56:31.952 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.943+0000 7f1f34ecb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f300b5bf0 0x7f1f30067000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.952 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.943+0000 7f1f34ecb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f300b5bf0 0x7f1f30067000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54820/0 (socket says 172.21.15.40:54820) 2020-11-20T13:56:31.952 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.943+0000 7f1f34ecb700 1 -- 172.21.15.40:0/373329080 learned_addr learned my addr 172.21.15.40:0/373329080 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:31.952 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.943+0000 7f1f356cc700 1 --2- 172.21.15.40:0/373329080 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1f300b3960 0x7f1f30140c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.952 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.944+0000 7f1f35ecd700 1 --2- 172.21.15.40:0/373329080 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f30144390 0x7f1f30135dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.953 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.944+0000 7f1f34ecb700 1 -- 172.21.15.40:0/373329080 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1f300b3960 msgr2=0x7f1f30140c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.953 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.944+0000 7f1f34ecb700 1 --2- 172.21.15.40:0/373329080 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1f300b3960 0x7f1f30140c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.953 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.944+0000 7f1f34ecb700 1 -- 172.21.15.40:0/373329080 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f30144390 msgr2=0x7f1f30135dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.953 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.944+0000 7f1f34ecb700 1 --2- 172.21.15.40:0/373329080 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f30144390 0x7f1f30135dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.953 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.944+0000 7f1f34ecb700 1 -- 172.21.15.40:0/373329080 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f300afde0 con 0x7f1f300b5bf0 2020-11-20T13:56:31.954 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.944+0000 7f1f34ecb700 1 --2- 172.21.15.40:0/373329080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f300b5bf0 0x7f1f30067000 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f1f2c000e90 tx=0x7f1f2c007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:31.954 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.944+0000 7f1f227fc700 1 -- 172.21.15.40:0/373329080 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f2c007320 con 0x7f1f300b5bf0 2020-11-20T13:56:31.954 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.944+0000 7f1f227fc700 1 -- 172.21.15.40:0/373329080 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f2c007480 con 0x7f1f300b5bf0 2020-11-20T13:56:31.954 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.944+0000 7f1f227fc700 1 -- 172.21.15.40:0/373329080 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f2c0057d0 con 0x7f1f300b5bf0 2020-11-20T13:56:31.955 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.944+0000 7f1f37930700 1 -- 172.21.15.40:0/373329080 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f300ad700 con 0x7f1f300b5bf0 2020-11-20T13:56:31.955 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.944+0000 7f1f37930700 1 -- 172.21.15.40:0/373329080 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f30136470 con 0x7f1f300b5bf0 2020-11-20T13:56:31.955 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.945+0000 7f1f227fc700 1 -- 172.21.15.40:0/373329080 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f1f2c010030 con 0x7f1f300b5bf0 2020-11-20T13:56:31.955 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.945+0000 7f1f37930700 1 -- 172.21.15.40:0/373329080 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f300afb00 con 0x7f1f300b5bf0 2020-11-20T13:56:31.955 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.945+0000 7f1f227fc700 1 --2- 172.21.15.40:0/373329080 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f1f10051290 0x7f1f10053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.955 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.945+0000 7f1f227fc700 1 -- 172.21.15.40:0/373329080 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f1f2c068e40 con 0x7f1f300b5bf0 2020-11-20T13:56:31.956 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.945+0000 7f1f356cc700 1 --2- 172.21.15.40:0/373329080 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f1f10051290 0x7f1f10053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.956 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.946+0000 7f1f356cc700 1 --2- 172.21.15.40:0/373329080 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f1f10051290 0x7f1f10053750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f1f24006000 tx=0x7f1f24008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:31.956 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:31.949+0000 7f1f227fc700 1 -- 172.21.15.40:0/373329080 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f1f2c041060 con 0x7f1f300b5bf0 2020-11-20T13:56:32.112 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:32.111+0000 7f1f37930700 1 -- 172.21.15.40:0/373329080 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1f30057050 con 0x7f1f300b5bf0 2020-11-20T13:56:32.112 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:32.111+0000 7f1f227fc700 1 -- 172.21.15.40:0/373329080 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1f2c044680 con 0x7f1f300b5bf0 2020-11-20T13:56:32.113 INFO:teuthology.orchestra.run.smithi040.stdout:55834574852 2020-11-20T13:56:32.114 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:32.113+0000 7f1f37930700 1 -- 172.21.15.40:0/373329080 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f1f10051290 msgr2=0x7f1f10053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:32.114 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:32.113+0000 7f1f37930700 1 --2- 172.21.15.40:0/373329080 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f1f10051290 0x7f1f10053750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f1f24006000 tx=0x7f1f24008040).stop 2020-11-20T13:56:32.115 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:32.113+0000 7f1f37930700 1 -- 172.21.15.40:0/373329080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f300b5bf0 msgr2=0x7f1f30067000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:32.115 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:32.113+0000 7f1f37930700 1 --2- 172.21.15.40:0/373329080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f300b5bf0 0x7f1f30067000 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f1f2c000e90 tx=0x7f1f2c007aa0).stop 2020-11-20T13:56:32.116 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:32.113+0000 7f1f37930700 1 -- 172.21.15.40:0/373329080 shutdown_connections 2020-11-20T13:56:32.116 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:32.113+0000 7f1f37930700 1 --2- 172.21.15.40:0/373329080 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f1f10051290 0x7f1f10053750 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:32.117 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:32.113+0000 7f1f37930700 1 --2- 172.21.15.40:0/373329080 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1f300b3960 0x7f1f30140c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:32.117 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:32.113+0000 7f1f37930700 1 --2- 172.21.15.40:0/373329080 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f30144390 0x7f1f30135dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:32.117 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:32.113+0000 7f1f37930700 1 --2- 172.21.15.40:0/373329080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f300b5bf0 0x7f1f30067000 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:32.118 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:32.113+0000 7f1f37930700 1 -- 172.21.15.40:0/373329080 >> 172.21.15.40:0/373329080 conn(0x7f1f300ba1d0 msgr2=0x7f1f30142810 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:32.118 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:32.113+0000 7f1f37930700 1 -- 172.21.15.40:0/373329080 shutdown_connections 2020-11-20T13:56:32.118 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:32.113+0000 7f1f37930700 1 -- 172.21.15.40:0/373329080 wait complete. 2020-11-20T13:56:32.123 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.0 2020-11-20T13:56:33.124 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.0 2020-11-20T13:56:33.254 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.254+0000 7fa25d023700 1 Processor -- start 2020-11-20T13:56:33.255 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.254+0000 7fa25d023700 1 -- start start 2020-11-20T13:56:33.257 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.255+0000 7fa25d023700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa2580b5bf0 0x7fa25805f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:33.257 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.255+0000 7fa25d023700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa258144390 0x7fa25805f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:33.257 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.255+0000 7fa25d023700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa25805fd70 0x7fa2580b6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:33.257 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.255+0000 7fa25d023700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2580b5fd0 con 0x7fa258144390 2020-11-20T13:56:33.258 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.255+0000 7fa25d023700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2580b05f0 con 0x7fa25805fd70 2020-11-20T13:56:33.258 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.255+0000 7fa25d023700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa258144870 con 0x7fa2580b5bf0 2020-11-20T13:56:33.258 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.255+0000 7fa24ffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa258144390 0x7fa25805f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:33.259 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.255+0000 7fa24ffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa258144390 0x7fa25805f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54826/0 (socket says 172.21.15.40:54826) 2020-11-20T13:56:33.259 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.255+0000 7fa24ffff700 1 -- 172.21.15.40:0/46136798 learned_addr learned my addr 172.21.15.40:0/46136798 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:33.259 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.255+0000 7fa256d9d700 1 --2- 172.21.15.40:0/46136798 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa2580b5bf0 0x7fa25805f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:33.259 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.255+0000 7fa25759e700 1 --2- 172.21.15.40:0/46136798 >> v2:172.21.15.116:3300/0 conn(0x7fa25805fd70 0x7fa2580b6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:33.259 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.255+0000 7fa24ffff700 1 -- 172.21.15.40:0/46136798 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa2580b5bf0 msgr2=0x7fa25805f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:33.260 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.255+0000 7fa24ffff700 1 --2- 172.21.15.40:0/46136798 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa2580b5bf0 0x7fa25805f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.260 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.255+0000 7fa24ffff700 1 -- 172.21.15.40:0/46136798 >> v2:172.21.15.116:3300/0 conn(0x7fa25805fd70 msgr2=0x7fa2580b6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:33.260 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.255+0000 7fa24ffff700 1 --2- 172.21.15.40:0/46136798 >> v2:172.21.15.116:3300/0 conn(0x7fa25805fd70 0x7fa2580b6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.260 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.255+0000 7fa24ffff700 1 -- 172.21.15.40:0/46136798 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2580b0c30 con 0x7fa258144390 2020-11-20T13:56:33.260 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.256+0000 7fa24ffff700 1 --2- 172.21.15.40:0/46136798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa258144390 0x7fa25805f880 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fa2580b2780 tx=0x7fa240006e80).ready entity=mon.0 client_cookie=fb31662863921501 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:33.261 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.256+0000 7fa25659c700 1 -- 172.21.15.40:0/46136798 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa24000be60 con 0x7fa258144390 2020-11-20T13:56:33.261 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.256+0000 7fa25659c700 1 -- 172.21.15.40:0/46136798 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2400097b0 con 0x7fa258144390 2020-11-20T13:56:33.261 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.256+0000 7fa25659c700 1 -- 172.21.15.40:0/46136798 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa240009b10 con 0x7fa258144390 2020-11-20T13:56:33.261 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.256+0000 7fa25d023700 1 -- 172.21.15.40:0/46136798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa258144390 msgr2=0x7fa25805f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:33.262 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.256+0000 7fa25d023700 1 --2- 172.21.15.40:0/46136798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa258144390 0x7fa25805f880 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fa2580b2780 tx=0x7fa240006e80).stop 2020-11-20T13:56:33.262 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.256+0000 7fa25d023700 1 -- 172.21.15.40:0/46136798 shutdown_connections 2020-11-20T13:56:33.262 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.256+0000 7fa25d023700 1 --2- 172.21.15.40:0/46136798 >> v2:172.21.15.116:3300/0 conn(0x7fa25805fd70 0x7fa2580b6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.262 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.256+0000 7fa25d023700 1 --2- 172.21.15.40:0/46136798 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa2580b5bf0 0x7fa25805f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.263 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.256+0000 7fa25d023700 1 --2- 172.21.15.40:0/46136798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa258144390 0x7fa25805f880 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.263 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.256+0000 7fa25d023700 1 -- 172.21.15.40:0/46136798 >> 172.21.15.40:0/46136798 conn(0x7fa2580ba1d0 msgr2=0x7fa2580b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:33.263 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.256+0000 7fa25d023700 1 -- 172.21.15.40:0/46136798 shutdown_connections 2020-11-20T13:56:33.263 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.256+0000 7fa25d023700 1 -- 172.21.15.40:0/46136798 wait complete. 2020-11-20T13:56:33.267 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.257+0000 7fa25d023700 1 Processor -- start 2020-11-20T13:56:33.267 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.257+0000 7fa25d023700 1 -- start start 2020-11-20T13:56:33.267 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.258+0000 7fa25d023700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa25805fd70 0x7fa2580adf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:33.267 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.258+0000 7fa25d023700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2580b5bf0 0x7fa258136ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:33.268 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.258+0000 7fa25d023700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa258144390 0x7fa25813fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:33.268 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.258+0000 7fa25d023700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa258157930 con 0x7fa2580b5bf0 2020-11-20T13:56:33.269 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.258+0000 7fa25d023700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25813f7b0 con 0x7fa25805fd70 2020-11-20T13:56:33.269 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.258+0000 7fa25d023700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa25813fa50 con 0x7fa258144390 2020-11-20T13:56:33.269 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.258+0000 7fa24ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2580b5bf0 0x7fa258136ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:33.270 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.258+0000 7fa24ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2580b5bf0 0x7fa258136ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54832/0 (socket says 172.21.15.40:54832) 2020-11-20T13:56:33.270 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.258+0000 7fa24ffff700 1 -- 172.21.15.40:0/3305756137 learned_addr learned my addr 172.21.15.40:0/3305756137 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:33.270 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.258+0000 7fa256d9d700 1 --2- 172.21.15.40:0/3305756137 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa25805fd70 0x7fa2580adf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:33.270 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.258+0000 7fa25759e700 1 --2- 172.21.15.40:0/3305756137 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa258144390 0x7fa25813fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:33.270 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.259+0000 7fa24ffff700 1 -- 172.21.15.40:0/3305756137 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa258144390 msgr2=0x7fa25813fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:33.271 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.259+0000 7fa24ffff700 1 --2- 172.21.15.40:0/3305756137 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa258144390 0x7fa25813fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.271 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.259+0000 7fa24ffff700 1 -- 172.21.15.40:0/3305756137 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa25805fd70 msgr2=0x7fa2580adf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:33.271 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.259+0000 7fa24ffff700 1 --2- 172.21.15.40:0/3305756137 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa25805fd70 0x7fa2580adf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.271 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.259+0000 7fa24ffff700 1 -- 172.21.15.40:0/3305756137 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa258157620 con 0x7fa2580b5bf0 2020-11-20T13:56:33.271 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.259+0000 7fa24ffff700 1 --2- 172.21.15.40:0/3305756137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2580b5bf0 0x7fa258136ad0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fa2580b2780 tx=0x7fa240015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:33.272 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.259+0000 7fa254d99700 1 -- 172.21.15.40:0/3305756137 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa24000be60 con 0x7fa2580b5bf0 2020-11-20T13:56:33.272 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.259+0000 7fa254d99700 1 -- 172.21.15.40:0/3305756137 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa240011b80 con 0x7fa2580b5bf0 2020-11-20T13:56:33.272 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.259+0000 7fa25d023700 1 -- 172.21.15.40:0/3305756137 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2580af7a0 con 0x7fa2580b5bf0 2020-11-20T13:56:33.272 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.259+0000 7fa254d99700 1 -- 172.21.15.40:0/3305756137 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa24001e460 con 0x7fa2580b5bf0 2020-11-20T13:56:33.272 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.259+0000 7fa25d023700 1 -- 172.21.15.40:0/3305756137 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa258137180 con 0x7fa2580b5bf0 2020-11-20T13:56:33.273 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.260+0000 7fa25d023700 1 -- 172.21.15.40:0/3305756137 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa25813fc10 con 0x7fa2580b5bf0 2020-11-20T13:56:33.273 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.260+0000 7fa254d99700 1 -- 172.21.15.40:0/3305756137 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fa240025030 con 0x7fa2580b5bf0 2020-11-20T13:56:33.273 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.260+0000 7fa254d99700 1 --2- 172.21.15.40:0/3305756137 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa23c051320 0x7fa23c0537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:33.273 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.260+0000 7fa254d99700 1 -- 172.21.15.40:0/3305756137 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fa24006a2d0 con 0x7fa2580b5bf0 2020-11-20T13:56:33.273 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.261+0000 7fa256d9d700 1 --2- 172.21.15.40:0/3305756137 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa23c051320 0x7fa23c0537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:33.274 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.261+0000 7fa256d9d700 1 --2- 172.21.15.40:0/3305756137 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa23c051320 0x7fa23c0537e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fa248006000 tx=0x7fa248006d60).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:33.275 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.264+0000 7fa254d99700 1 -- 172.21.15.40:0/3305756137 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fa2400424f0 con 0x7fa2580b5bf0 2020-11-20T13:56:33.423 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.423+0000 7fa25d023700 1 -- 172.21.15.40:0/3305756137 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa258057050 con 0x7fa2580b5bf0 2020-11-20T13:56:33.424 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.424+0000 7fa254d99700 1 -- 172.21.15.40:0/3305756137 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa240023020 con 0x7fa2580b5bf0 2020-11-20T13:56:33.424 INFO:teuthology.orchestra.run.smithi040.stdout:55834574853 2020-11-20T13:56:33.425 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.425+0000 7fa25d023700 1 -- 172.21.15.40:0/3305756137 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa23c051320 msgr2=0x7fa23c0537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:33.425 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.425+0000 7fa25d023700 1 --2- 172.21.15.40:0/3305756137 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa23c051320 0x7fa23c0537e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fa248006000 tx=0x7fa248006d60).stop 2020-11-20T13:56:33.426 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.425+0000 7fa25d023700 1 -- 172.21.15.40:0/3305756137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2580b5bf0 msgr2=0x7fa258136ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:33.426 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.425+0000 7fa25d023700 1 --2- 172.21.15.40:0/3305756137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2580b5bf0 0x7fa258136ad0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fa2580b2780 tx=0x7fa240015000).stop 2020-11-20T13:56:33.426 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.425+0000 7fa25d023700 1 -- 172.21.15.40:0/3305756137 shutdown_connections 2020-11-20T13:56:33.426 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.425+0000 7fa25d023700 1 --2- 172.21.15.40:0/3305756137 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa23c051320 0x7fa23c0537e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.426 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.425+0000 7fa25d023700 1 --2- 172.21.15.40:0/3305756137 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa258144390 0x7fa25813fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.427 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.425+0000 7fa25d023700 1 --2- 172.21.15.40:0/3305756137 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa25805fd70 0x7fa2580adf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.427 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.425+0000 7fa25d023700 1 --2- 172.21.15.40:0/3305756137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2580b5bf0 0x7fa258136ad0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.427 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.425+0000 7fa25d023700 1 -- 172.21.15.40:0/3305756137 >> 172.21.15.40:0/3305756137 conn(0x7fa2580ba1d0 msgr2=0x7fa2580629a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:33.427 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.425+0000 7fa25d023700 1 -- 172.21.15.40:0/3305756137 shutdown_connections 2020-11-20T13:56:33.428 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.425+0000 7fa25d023700 1 -- 172.21.15.40:0/3305756137 wait complete. 2020-11-20T13:56:33.436 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.0 2020-11-20T13:56:33.436 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.1 2020-11-20T13:56:33.552 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.551+0000 7f5007faa700 1 Processor -- start 2020-11-20T13:56:33.552 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.552+0000 7f5007faa700 1 -- start start 2020-11-20T13:56:33.554 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.552+0000 7f5007faa700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f50000b31f0 0x7f5000144bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:33.554 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.552+0000 7f5007faa700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f50000b1450 0x7f50000b1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:33.554 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.552+0000 7f5007faa700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50000b1df0 con 0x7f50000b6c80 2020-11-20T13:56:33.555 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.552+0000 7f5007faa700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50000b0d10 con 0x7f50000b1450 2020-11-20T13:56:33.555 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.552+0000 7f5007faa700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50000b0fb0 con 0x7f50000b31f0 2020-11-20T13:56:33.555 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.552+0000 7f5006547700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f50000b1450 0x7f50000b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:33.555 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.552+0000 7f5005d46700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f50000b6c80 0x7f50000b40f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45434/0 (socket says 172.21.15.40:45434) 2020-11-20T13:56:33.555 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.553+0000 7f5005d46700 1 -- 172.21.15.40:0/3927005864 learned_addr learned my addr 172.21.15.40:0/3927005864 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:33.556 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.553+0000 7f5005545700 1 --2- 172.21.15.40:0/3927005864 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f50000b31f0 0x7f5000144bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:33.556 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.553+0000 7f5006547700 1 -- 172.21.15.40:0/3927005864 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f50000b31f0 msgr2=0x7f5000144bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:33.556 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.553+0000 7f5006547700 1 --2- 172.21.15.40:0/3927005864 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f50000b31f0 0x7f5000144bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.556 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.553+0000 7f5006547700 1 -- 172.21.15.40:0/3927005864 >> v1:172.21.15.40:6789/0 conn(0x7f50000b6c80 legacy=0x7f50000b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:33.556 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.553+0000 7f5006547700 1 -- 172.21.15.40:0/3927005864 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50000b7060 con 0x7f50000b1450 2020-11-20T13:56:33.557 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.553+0000 7f5004d44700 1 -- 172.21.15.40:0/3927005864 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4224770898 0 0) 0x7f50000b1df0 con 0x7f50000b6c80 2020-11-20T13:56:33.557 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.553+0000 7f5006547700 1 --2- 172.21.15.40:0/3927005864 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50000b1450 0x7f50000b1900 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f50000b3100 tx=0x7f4ff80091a0).ready entity=mon.0 client_cookie=c74e5c4fb5e6ba68 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:33.558 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.553+0000 7f5004d44700 1 -- 172.21.15.40:0/3927005864 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ff8009e70 con 0x7f50000b1450 2020-11-20T13:56:33.558 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.553+0000 7f5004d44700 1 -- 172.21.15.40:0/3927005864 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff800bbc0 con 0x7f50000b1450 2020-11-20T13:56:33.558 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.553+0000 7f5004d44700 1 -- 172.21.15.40:0/3927005864 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ff800bd30 con 0x7f50000b1450 2020-11-20T13:56:33.559 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.553+0000 7f5007faa700 1 -- 172.21.15.40:0/3927005864 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50000b1450 msgr2=0x7f50000b1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:33.559 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.553+0000 7f5007faa700 1 --2- 172.21.15.40:0/3927005864 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50000b1450 0x7f50000b1900 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f50000b3100 tx=0x7f4ff80091a0).stop 2020-11-20T13:56:33.559 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.554+0000 7f5007faa700 1 -- 172.21.15.40:0/3927005864 shutdown_connections 2020-11-20T13:56:33.559 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.554+0000 7f5007faa700 1 --2- 172.21.15.40:0/3927005864 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50000b1450 0x7f50000b1900 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.559 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.554+0000 7f5007faa700 1 --2- 172.21.15.40:0/3927005864 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f50000b31f0 0x7f5000144bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.560 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.554+0000 7f5007faa700 1 -- 172.21.15.40:0/3927005864 >> 172.21.15.40:0/3927005864 conn(0x7f50000ba1d0 msgr2=0x7f50000b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:33.560 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.554+0000 7f5007faa700 1 -- 172.21.15.40:0/3927005864 shutdown_connections 2020-11-20T13:56:33.560 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.554+0000 7f5007faa700 1 -- 172.21.15.40:0/3927005864 wait complete. 2020-11-20T13:56:33.563 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.555+0000 7f5007faa700 1 Processor -- start 2020-11-20T13:56:33.563 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.555+0000 7f5007faa700 1 -- start start 2020-11-20T13:56:33.563 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.555+0000 7f5007faa700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f50000b1450 0x7f5000148050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:33.564 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.555+0000 7f5007faa700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50000b31f0 0x7f500005f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:33.564 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.555+0000 7f5007faa700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50000b6c80 0x7f500005f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:33.564 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.555+0000 7f5007faa700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5000157c00 con 0x7f50000b31f0 2020-11-20T13:56:33.564 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.555+0000 7f5007faa700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5000157ea0 con 0x7f50000b6c80 2020-11-20T13:56:33.564 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.555+0000 7f5007faa700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5000158140 con 0x7f50000b1450 2020-11-20T13:56:33.565 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.555+0000 7f5005545700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50000b31f0 0x7f500005f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:33.565 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.555+0000 7f5005545700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50000b31f0 0x7f500005f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54844/0 (socket says 172.21.15.40:54844) 2020-11-20T13:56:33.565 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.555+0000 7f5005545700 1 -- 172.21.15.40:0/4000093896 learned_addr learned my addr 172.21.15.40:0/4000093896 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:33.565 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.556+0000 7f5005d46700 1 --2- 172.21.15.40:0/4000093896 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f50000b1450 0x7f5000148050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:33.565 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.556+0000 7f5006547700 1 --2- 172.21.15.40:0/4000093896 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50000b6c80 0x7f500005f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:33.566 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.556+0000 7f5005545700 1 -- 172.21.15.40:0/4000093896 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f50000b1450 msgr2=0x7f5000148050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:33.566 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.556+0000 7f5005545700 1 --2- 172.21.15.40:0/4000093896 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f50000b1450 0x7f5000148050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.566 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.556+0000 7f5005545700 1 -- 172.21.15.40:0/4000093896 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50000b6c80 msgr2=0x7f500005f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:33.566 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.556+0000 7f5005545700 1 --2- 172.21.15.40:0/4000093896 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50000b6c80 0x7f500005f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.566 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.556+0000 7f5005545700 1 -- 172.21.15.40:0/4000093896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5000157880 con 0x7f50000b31f0 2020-11-20T13:56:33.567 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.556+0000 7f5005545700 1 --2- 172.21.15.40:0/4000093896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50000b31f0 0x7f500005f390 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f4ffc00a000 tx=0x7f4ffc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:33.567 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.556+0000 7f4ff2ffd700 1 -- 172.21.15.40:0/4000093896 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ffc009790 con 0x7f50000b31f0 2020-11-20T13:56:33.567 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.556+0000 7f4ff2ffd700 1 -- 172.21.15.40:0/4000093896 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ffc0098f0 con 0x7f50000b31f0 2020-11-20T13:56:33.567 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.556+0000 7f5007faa700 1 -- 172.21.15.40:0/4000093896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50001585e0 con 0x7f50000b31f0 2020-11-20T13:56:33.568 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.556+0000 7f4ff2ffd700 1 -- 172.21.15.40:0/4000093896 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ffc009c50 con 0x7f50000b31f0 2020-11-20T13:56:33.568 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.556+0000 7f5007faa700 1 -- 172.21.15.40:0/4000093896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50000659b0 con 0x7f50000b31f0 2020-11-20T13:56:33.568 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.557+0000 7f5007faa700 1 -- 172.21.15.40:0/4000093896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5000057050 con 0x7f50000b31f0 2020-11-20T13:56:33.568 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.557+0000 7f4ff2ffd700 1 -- 172.21.15.40:0/4000093896 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f4ffc013480 con 0x7f50000b31f0 2020-11-20T13:56:33.569 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.558+0000 7f4ff2ffd700 1 --2- 172.21.15.40:0/4000093896 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f4fe0051320 0x7f4fe00537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:33.569 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.558+0000 7f4ff2ffd700 1 -- 172.21.15.40:0/4000093896 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f4ffc0697f0 con 0x7f50000b31f0 2020-11-20T13:56:33.569 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.558+0000 7f5005d46700 1 --2- 172.21.15.40:0/4000093896 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f4fe0051320 0x7f4fe00537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:33.569 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.558+0000 7f5005d46700 1 --2- 172.21.15.40:0/4000093896 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f4fe0051320 0x7f4fe00537e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f4ff400d000 tx=0x7f4ff40016b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:33.570 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.561+0000 7f4ff2ffd700 1 -- 172.21.15.40:0/4000093896 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f4ffc0419b0 con 0x7f50000b31f0 2020-11-20T13:56:33.723 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.721+0000 7f5007faa700 1 -- 172.21.15.40:0/4000093896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f500005edb0 con 0x7f50000b31f0 2020-11-20T13:56:33.724 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.721+0000 7f4ff2ffd700 1 -- 172.21.15.40:0/4000093896 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4ffc009db0 con 0x7f50000b31f0 2020-11-20T13:56:33.724 INFO:teuthology.orchestra.run.smithi040.stdout:55834574854 2020-11-20T13:56:33.725 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.723+0000 7f5007faa700 1 -- 172.21.15.40:0/4000093896 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f4fe0051320 msgr2=0x7f4fe00537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:33.726 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.723+0000 7f5007faa700 1 --2- 172.21.15.40:0/4000093896 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f4fe0051320 0x7f4fe00537e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f4ff400d000 tx=0x7f4ff40016b0).stop 2020-11-20T13:56:33.726 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.723+0000 7f5007faa700 1 -- 172.21.15.40:0/4000093896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50000b31f0 msgr2=0x7f500005f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:33.726 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.723+0000 7f5007faa700 1 --2- 172.21.15.40:0/4000093896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50000b31f0 0x7f500005f390 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f4ffc00a000 tx=0x7f4ffc008040).stop 2020-11-20T13:56:33.726 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.723+0000 7f5007faa700 1 -- 172.21.15.40:0/4000093896 shutdown_connections 2020-11-20T13:56:33.727 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.723+0000 7f5007faa700 1 --2- 172.21.15.40:0/4000093896 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f4fe0051320 0x7f4fe00537e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.727 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.723+0000 7f5007faa700 1 --2- 172.21.15.40:0/4000093896 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f50000b1450 0x7f5000148050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.727 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.723+0000 7f5007faa700 1 --2- 172.21.15.40:0/4000093896 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50000b6c80 0x7f500005f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.728 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.723+0000 7f5007faa700 1 --2- 172.21.15.40:0/4000093896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50000b31f0 0x7f500005f390 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.729 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.723+0000 7f5007faa700 1 -- 172.21.15.40:0/4000093896 >> 172.21.15.40:0/4000093896 conn(0x7f50000ba1d0 msgr2=0x7f5000144680 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:33.730 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.723+0000 7f5007faa700 1 -- 172.21.15.40:0/4000093896 shutdown_connections 2020-11-20T13:56:33.730 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.723+0000 7f5007faa700 1 -- 172.21.15.40:0/4000093896 wait complete. 2020-11-20T13:56:33.733 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2020-11-20T13:56:33.733 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.2 2020-11-20T13:56:33.855 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.848+0000 7f091167c700 1 Processor -- start 2020-11-20T13:56:33.856 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.848+0000 7f091167c700 1 -- start start 2020-11-20T13:56:33.856 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.849+0000 7f091167c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f090c05c510 0x7f090c14b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:33.856 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.849+0000 7f091167c700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f090c0b6980 0x7f090c0b6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:33.857 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.849+0000 7f091167c700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f090c061310 con 0x7f090c061f90 2020-11-20T13:56:33.857 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.849+0000 7f091167c700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f090c0615b0 con 0x7f090c05c510 2020-11-20T13:56:33.857 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.849+0000 7f091167c700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f090c061850 con 0x7f090c0b6980 2020-11-20T13:56:33.857 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.849+0000 7f090affd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f090c05c510 0x7f090c14b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:33.857 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.849+0000 7f090affd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f090c05c510 0x7f090c14b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54852/0 (socket says 172.21.15.40:54852) 2020-11-20T13:56:33.858 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.849+0000 7f090affd700 1 -- 172.21.15.40:0/2253103790 learned_addr learned my addr 172.21.15.40:0/2253103790 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:33.858 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.849+0000 7f090b7fe700 1 --2- 172.21.15.40:0/2253103790 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f090c0b6980 0x7f090c0b6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:33.858 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.850+0000 7f090affd700 1 -- 172.21.15.40:0/2253103790 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f090c0b6980 msgr2=0x7f090c0b6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:33.858 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.850+0000 7f090affd700 1 --2- 172.21.15.40:0/2253103790 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f090c0b6980 0x7f090c0b6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.858 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.850+0000 7f090affd700 1 -- 172.21.15.40:0/2253103790 >> v1:172.21.15.116:6789/0 conn(0x7f090c061f90 legacy=0x7f090c12d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:33.859 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.850+0000 7f090affd700 1 -- 172.21.15.40:0/2253103790 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f090c061af0 con 0x7f090c05c510 2020-11-20T13:56:33.859 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.850+0000 7f090affd700 1 --2- 172.21.15.40:0/2253103790 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f090c05c510 0x7f090c14b4a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f0900002f30 tx=0x7f0900006e80).ready entity=mon.0 client_cookie=5f9c0789faf7348f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:33.859 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.850+0000 7f0909ffb700 1 -- 172.21.15.40:0/2253103790 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09000097f0 con 0x7f090c05c510 2020-11-20T13:56:33.859 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.850+0000 7f0909ffb700 1 -- 172.21.15.40:0/2253103790 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0900009950 con 0x7f090c05c510 2020-11-20T13:56:33.859 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.850+0000 7f0909ffb700 1 -- 172.21.15.40:0/2253103790 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0900009cb0 con 0x7f090c05c510 2020-11-20T13:56:33.860 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.850+0000 7f091167c700 1 -- 172.21.15.40:0/2253103790 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f090c05c510 msgr2=0x7f090c14b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:33.860 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.850+0000 7f091167c700 1 --2- 172.21.15.40:0/2253103790 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f090c05c510 0x7f090c14b4a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f0900002f30 tx=0x7f0900006e80).stop 2020-11-20T13:56:33.860 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.850+0000 7f091167c700 1 -- 172.21.15.40:0/2253103790 shutdown_connections 2020-11-20T13:56:33.860 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.850+0000 7f091167c700 1 --2- 172.21.15.40:0/2253103790 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f090c0b6980 0x7f090c0b6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.860 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.850+0000 7f091167c700 1 --2- 172.21.15.40:0/2253103790 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f090c05c510 0x7f090c14b4a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.861 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.850+0000 7f091167c700 1 -- 172.21.15.40:0/2253103790 >> 172.21.15.40:0/2253103790 conn(0x7f090c0bdb70 msgr2=0x7f090c0bdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:33.861 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.850+0000 7f091167c700 1 -- 172.21.15.40:0/2253103790 shutdown_connections 2020-11-20T13:56:33.861 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.850+0000 7f091167c700 1 -- 172.21.15.40:0/2253103790 wait complete. 2020-11-20T13:56:33.861 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.851+0000 7f091167c700 1 Processor -- start 2020-11-20T13:56:33.861 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.851+0000 7f091167c700 1 -- start start 2020-11-20T13:56:33.862 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.852+0000 7f091167c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f090c05c510 0x7f090c144470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:33.862 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.852+0000 7f091167c700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f090c061f90 0x7f090c143110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:33.862 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.852+0000 7f091167c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f090c0b6980 0x7f090c143600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:33.862 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.852+0000 7f091167c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090c157640 con 0x7f090c0b6980 2020-11-20T13:56:33.862 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.852+0000 7f091167c700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090c156c60 con 0x7f090c05c510 2020-11-20T13:56:33.863 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.852+0000 7f091167c700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f090c156f00 con 0x7f090c061f90 2020-11-20T13:56:33.863 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.852+0000 7f090b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f090c0b6980 0x7f090c143600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:33.863 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.852+0000 7f090b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f090c0b6980 0x7f090c143600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54862/0 (socket says 172.21.15.40:54862) 2020-11-20T13:56:33.863 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.852+0000 7f090b7fe700 1 -- 172.21.15.40:0/4064248586 learned_addr learned my addr 172.21.15.40:0/4064248586 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:33.863 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.852+0000 7f090affd700 1 --2- 172.21.15.40:0/4064248586 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f090c05c510 0x7f090c144470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:33.864 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.852+0000 7f090b7fe700 1 -- 172.21.15.40:0/4064248586 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f090c061f90 msgr2=0x7f090c143110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:33.864 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.852+0000 7f090a7fc700 1 --2- 172.21.15.40:0/4064248586 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f090c061f90 0x7f090c143110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:33.864 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.852+0000 7f090b7fe700 1 --2- 172.21.15.40:0/4064248586 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f090c061f90 0x7f090c143110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.864 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.853+0000 7f090b7fe700 1 -- 172.21.15.40:0/4064248586 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f090c05c510 msgr2=0x7f090c144470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:33.864 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.853+0000 7f090b7fe700 1 --2- 172.21.15.40:0/4064248586 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f090c05c510 0x7f090c144470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.865 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.853+0000 7f090b7fe700 1 -- 172.21.15.40:0/4064248586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f090c1571a0 con 0x7f090c0b6980 2020-11-20T13:56:33.865 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.853+0000 7f090affd700 1 --2- 172.21.15.40:0/4064248586 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f090c05c510 0x7f090c144470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:33.865 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.853+0000 7f090b7fe700 1 --2- 172.21.15.40:0/4064248586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f090c0b6980 0x7f090c143600 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f0904006dd0 tx=0x7f09040091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:33.865 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.853+0000 7f08fbfff700 1 -- 172.21.15.40:0/4064248586 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f090400eed0 con 0x7f090c0b6980 2020-11-20T13:56:33.865 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.853+0000 7f08fbfff700 1 -- 172.21.15.40:0/4064248586 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f090400cbb0 con 0x7f090c0b6980 2020-11-20T13:56:33.866 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.853+0000 7f08fbfff700 1 -- 172.21.15.40:0/4064248586 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09040123f0 con 0x7f090c0b6980 2020-11-20T13:56:33.866 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.853+0000 7f091167c700 1 -- 172.21.15.40:0/4064248586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f090c15aac0 con 0x7f090c0b6980 2020-11-20T13:56:33.866 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.853+0000 7f091167c700 1 -- 172.21.15.40:0/4064248586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f090c156950 con 0x7f090c0b6980 2020-11-20T13:56:33.866 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.854+0000 7f091167c700 1 -- 172.21.15.40:0/4064248586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f090c057050 con 0x7f090c0b6980 2020-11-20T13:56:33.866 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.854+0000 7f08fbfff700 1 -- 172.21.15.40:0/4064248586 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f0904022070 con 0x7f090c0b6980 2020-11-20T13:56:33.867 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.854+0000 7f08fbfff700 1 --2- 172.21.15.40:0/4064248586 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f08e80512d0 0x7f08e8053790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:33.867 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.854+0000 7f08fbfff700 1 -- 172.21.15.40:0/4064248586 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f0904069860 con 0x7f090c0b6980 2020-11-20T13:56:33.867 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.854+0000 7f090affd700 1 --2- 172.21.15.40:0/4064248586 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f08e80512d0 0x7f08e8053790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:33.867 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.855+0000 7f090affd700 1 --2- 172.21.15.40:0/4064248586 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f08e80512d0 0x7f08e8053790 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f0900017000 tx=0x7f0900015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:33.868 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:33.858+0000 7f08fbfff700 1 -- 172.21.15.40:0/4064248586 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f0904041a80 con 0x7f090c0b6980 2020-11-20T13:56:34.018 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.018+0000 7f091167c700 1 -- 172.21.15.40:0/4064248586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f090c05c8f0 con 0x7f090c0b6980 2020-11-20T13:56:34.019 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.018+0000 7f08fbfff700 1 -- 172.21.15.40:0/4064248586 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0904016070 con 0x7f090c0b6980 2020-11-20T13:56:34.019 INFO:teuthology.orchestra.run.smithi040.stdout:55834574853 2020-11-20T13:56:34.021 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.020+0000 7f091167c700 1 -- 172.21.15.40:0/4064248586 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f08e80512d0 msgr2=0x7f08e8053790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:34.021 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.020+0000 7f091167c700 1 --2- 172.21.15.40:0/4064248586 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f08e80512d0 0x7f08e8053790 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f0900017000 tx=0x7f0900015000).stop 2020-11-20T13:56:34.022 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.020+0000 7f091167c700 1 -- 172.21.15.40:0/4064248586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f090c0b6980 msgr2=0x7f090c143600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:34.022 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.020+0000 7f091167c700 1 --2- 172.21.15.40:0/4064248586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f090c0b6980 0x7f090c143600 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f0904006dd0 tx=0x7f09040091a0).stop 2020-11-20T13:56:34.022 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.020+0000 7f091167c700 1 -- 172.21.15.40:0/4064248586 shutdown_connections 2020-11-20T13:56:34.022 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.020+0000 7f091167c700 1 --2- 172.21.15.40:0/4064248586 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f08e80512d0 0x7f08e8053790 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.022 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.020+0000 7f091167c700 1 --2- 172.21.15.40:0/4064248586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f090c0b6980 0x7f090c143600 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.023 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.020+0000 7f091167c700 1 --2- 172.21.15.40:0/4064248586 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f090c05c510 0x7f090c144470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.023 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.020+0000 7f091167c700 1 --2- 172.21.15.40:0/4064248586 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f090c061f90 0x7f090c143110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.023 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.020+0000 7f091167c700 1 -- 172.21.15.40:0/4064248586 >> 172.21.15.40:0/4064248586 conn(0x7f090c0bdb70 msgr2=0x7f090c0bba70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:34.023 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.020+0000 7f091167c700 1 -- 172.21.15.40:0/4064248586 shutdown_connections 2020-11-20T13:56:34.023 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.020+0000 7f091167c700 1 -- 172.21.15.40:0/4064248586 wait complete. 2020-11-20T13:56:34.031 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.2 2020-11-20T13:56:34.032 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.3 2020-11-20T13:56:34.147 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.147+0000 7f5b64d3b700 1 Processor -- start 2020-11-20T13:56:34.148 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.147+0000 7f5b64d3b700 1 -- start start 2020-11-20T13:56:34.160 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.148+0000 7f5b64d3b700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5b600b5bf0 0x7f5b6005f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:34.160 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.148+0000 7f5b64d3b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5b60144390 0x7f5b6005f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:34.160 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.148+0000 7f5b64d3b700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b600b5fd0 con 0x7f5b6005fd70 2020-11-20T13:56:34.161 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.148+0000 7f5b64d3b700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b600b05f0 con 0x7f5b60144390 2020-11-20T13:56:34.161 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.148+0000 7f5b64d3b700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b60144870 con 0x7f5b600b5bf0 2020-11-20T13:56:34.161 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.148+0000 7f5b5dd9b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5b60144390 0x7f5b6005f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:34.161 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.148+0000 7f5b5dd9b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5b60144390 0x7f5b6005f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54866/0 (socket says 172.21.15.40:54866) 2020-11-20T13:56:34.162 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.148+0000 7f5b5dd9b700 1 -- 172.21.15.40:0/3597624229 learned_addr learned my addr 172.21.15.40:0/3597624229 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:34.162 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.148+0000 7f5b5e59c700 1 --2- 172.21.15.40:0/3597624229 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5b600b5bf0 0x7f5b6005f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:34.162 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.149+0000 7f5b5dd9b700 1 -- 172.21.15.40:0/3597624229 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5b600b5bf0 msgr2=0x7f5b6005f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:34.162 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.149+0000 7f5b5dd9b700 1 --2- 172.21.15.40:0/3597624229 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5b600b5bf0 0x7f5b6005f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.162 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.149+0000 7f5b5dd9b700 1 -- 172.21.15.40:0/3597624229 >> v1:172.21.15.116:6789/0 conn(0x7f5b6005fd70 legacy=0x7f5b600b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:34.163 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.149+0000 7f5b5dd9b700 1 -- 172.21.15.40:0/3597624229 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b600b0c30 con 0x7f5b60144390 2020-11-20T13:56:34.163 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.149+0000 7f5b5dd9b700 1 --2- 172.21.15.40:0/3597624229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b60144390 0x7f5b6005f880 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f5b500032b0 tx=0x7f5b50009270).ready entity=mon.0 client_cookie=2473b0e6bcf7bed1 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:34.163 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.149+0000 7f5b5d59a700 1 -- 172.21.15.40:0/3597624229 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b5000bc00 con 0x7f5b60144390 2020-11-20T13:56:34.163 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.149+0000 7f5b5d59a700 1 -- 172.21.15.40:0/3597624229 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b5000bd60 con 0x7f5b60144390 2020-11-20T13:56:34.163 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.149+0000 7f5b5d59a700 1 -- 172.21.15.40:0/3597624229 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b50011400 con 0x7f5b60144390 2020-11-20T13:56:34.164 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.149+0000 7f5b64d3b700 1 -- 172.21.15.40:0/3597624229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b60144390 msgr2=0x7f5b6005f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:34.164 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.149+0000 7f5b64d3b700 1 --2- 172.21.15.40:0/3597624229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b60144390 0x7f5b6005f880 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f5b500032b0 tx=0x7f5b50009270).stop 2020-11-20T13:56:34.164 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.149+0000 7f5b64d3b700 1 -- 172.21.15.40:0/3597624229 shutdown_connections 2020-11-20T13:56:34.164 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.149+0000 7f5b64d3b700 1 --2- 172.21.15.40:0/3597624229 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5b600b5bf0 0x7f5b6005f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.164 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.149+0000 7f5b64d3b700 1 --2- 172.21.15.40:0/3597624229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b60144390 0x7f5b6005f880 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.165 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.149+0000 7f5b64d3b700 1 -- 172.21.15.40:0/3597624229 >> 172.21.15.40:0/3597624229 conn(0x7f5b600ba1d0 msgr2=0x7f5b600b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:34.165 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.149+0000 7f5b64d3b700 1 -- 172.21.15.40:0/3597624229 shutdown_connections 2020-11-20T13:56:34.165 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.149+0000 7f5b64d3b700 1 -- 172.21.15.40:0/3597624229 wait complete. 2020-11-20T13:56:34.165 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.150+0000 7f5b64d3b700 1 Processor -- start 2020-11-20T13:56:34.165 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.150+0000 7f5b64d3b700 1 -- start start 2020-11-20T13:56:34.166 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.151+0000 7f5b64d3b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b6005fd70 0x7f5b6013c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:34.166 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.151+0000 7f5b64d3b700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5b600b5bf0 0x7f5b600b2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:34.166 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.151+0000 7f5b64d3b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b60144390 0x7f5b60140250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:34.166 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.151+0000 7f5b64d3b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b60135960 con 0x7f5b6005fd70 2020-11-20T13:56:34.166 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.151+0000 7f5b64d3b700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b601587b0 con 0x7f5b60144390 2020-11-20T13:56:34.167 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.151+0000 7f5b64d3b700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b60157dd0 con 0x7f5b600b5bf0 2020-11-20T13:56:34.167 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.151+0000 7f5b5e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b6005fd70 0x7f5b6013c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:34.167 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.151+0000 7f5b5e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b6005fd70 0x7f5b6013c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54872/0 (socket says 172.21.15.40:54872) 2020-11-20T13:56:34.168 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.151+0000 7f5b5e59c700 1 -- 172.21.15.40:0/4017889590 learned_addr learned my addr 172.21.15.40:0/4017889590 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:34.168 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.151+0000 7f5b5dd9b700 1 --2- 172.21.15.40:0/4017889590 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5b600b5bf0 0x7f5b600b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:34.168 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.151+0000 7f5b5ed9d700 1 --2- 172.21.15.40:0/4017889590 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b60144390 0x7f5b60140250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:34.169 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.151+0000 7f5b5e59c700 1 -- 172.21.15.40:0/4017889590 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5b600b5bf0 msgr2=0x7f5b600b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:34.169 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.151+0000 7f5b5e59c700 1 --2- 172.21.15.40:0/4017889590 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5b600b5bf0 0x7f5b600b2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.169 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.151+0000 7f5b5e59c700 1 -- 172.21.15.40:0/4017889590 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b60144390 msgr2=0x7f5b60140250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:34.170 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.151+0000 7f5b5e59c700 1 --2- 172.21.15.40:0/4017889590 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b60144390 0x7f5b60140250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.170 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.151+0000 7f5b5e59c700 1 -- 172.21.15.40:0/4017889590 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b60158070 con 0x7f5b6005fd70 2020-11-20T13:56:34.170 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.152+0000 7f5b5e59c700 1 --2- 172.21.15.40:0/4017889590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b6005fd70 0x7f5b6013c3a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f5b60135aa0 tx=0x7f5b54008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:34.170 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.152+0000 7f5b4f7fe700 1 -- 172.21.15.40:0/4017889590 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b5400b130 con 0x7f5b6005fd70 2020-11-20T13:56:34.170 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.152+0000 7f5b4f7fe700 1 -- 172.21.15.40:0/4017889590 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b54009730 con 0x7f5b6005fd70 2020-11-20T13:56:34.171 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.152+0000 7f5b64d3b700 1 -- 172.21.15.40:0/4017889590 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b60158310 con 0x7f5b6005fd70 2020-11-20T13:56:34.171 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.152+0000 7f5b4f7fe700 1 -- 172.21.15.40:0/4017889590 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b54009a90 con 0x7f5b6005fd70 2020-11-20T13:56:34.171 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.152+0000 7f5b64d3b700 1 -- 172.21.15.40:0/4017889590 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b60157af0 con 0x7f5b6005fd70 2020-11-20T13:56:34.171 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.153+0000 7f5b4f7fe700 1 -- 172.21.15.40:0/4017889590 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f5b54009bf0 con 0x7f5b6005fd70 2020-11-20T13:56:34.171 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.153+0000 7f5b64d3b700 1 -- 172.21.15.40:0/4017889590 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b60057050 con 0x7f5b6005fd70 2020-11-20T13:56:34.172 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.153+0000 7f5b4f7fe700 1 --2- 172.21.15.40:0/4017889590 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f5b3c051320 0x7f5b3c0537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:34.172 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.153+0000 7f5b4f7fe700 1 -- 172.21.15.40:0/4017889590 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f5b54069510 con 0x7f5b6005fd70 2020-11-20T13:56:34.172 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.153+0000 7f5b5dd9b700 1 --2- 172.21.15.40:0/4017889590 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f5b3c051320 0x7f5b3c0537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:34.172 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.154+0000 7f5b5dd9b700 1 --2- 172.21.15.40:0/4017889590 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f5b3c051320 0x7f5b3c0537e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f5b5001b000 tx=0x7f5b50016000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:34.173 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.157+0000 7f5b4f7fe700 1 -- 172.21.15.40:0/4017889590 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f5b540415c0 con 0x7f5b6005fd70 2020-11-20T13:56:34.318 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.318+0000 7f5b64d3b700 1 -- 172.21.15.40:0/4017889590 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5b601584a0 con 0x7f5b6005fd70 2020-11-20T13:56:34.319 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.318+0000 7f5b4f7fe700 1 -- 172.21.15.40:0/4017889590 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5b54044be0 con 0x7f5b6005fd70 2020-11-20T13:56:34.319 INFO:teuthology.orchestra.run.smithi040.stdout:55834574854 2020-11-20T13:56:34.320 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.320+0000 7f5b64d3b700 1 -- 172.21.15.40:0/4017889590 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f5b3c051320 msgr2=0x7f5b3c0537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:34.320 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.320+0000 7f5b64d3b700 1 --2- 172.21.15.40:0/4017889590 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f5b3c051320 0x7f5b3c0537e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f5b5001b000 tx=0x7f5b50016000).stop 2020-11-20T13:56:34.321 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.320+0000 7f5b64d3b700 1 -- 172.21.15.40:0/4017889590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b6005fd70 msgr2=0x7f5b6013c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:34.321 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.320+0000 7f5b64d3b700 1 --2- 172.21.15.40:0/4017889590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b6005fd70 0x7f5b6013c3a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f5b60135aa0 tx=0x7f5b54008040).stop 2020-11-20T13:56:34.321 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.320+0000 7f5b64d3b700 1 -- 172.21.15.40:0/4017889590 shutdown_connections 2020-11-20T13:56:34.322 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.320+0000 7f5b64d3b700 1 --2- 172.21.15.40:0/4017889590 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f5b3c051320 0x7f5b3c0537e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.322 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.320+0000 7f5b64d3b700 1 --2- 172.21.15.40:0/4017889590 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b60144390 0x7f5b60140250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.322 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.320+0000 7f5b64d3b700 1 --2- 172.21.15.40:0/4017889590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b6005fd70 0x7f5b6013c3a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.322 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.320+0000 7f5b64d3b700 1 --2- 172.21.15.40:0/4017889590 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5b600b5bf0 0x7f5b600b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.322 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.320+0000 7f5b64d3b700 1 -- 172.21.15.40:0/4017889590 >> 172.21.15.40:0/4017889590 conn(0x7f5b600ba1d0 msgr2=0x7f5b60062870 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:34.323 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.320+0000 7f5b64d3b700 1 -- 172.21.15.40:0/4017889590 shutdown_connections 2020-11-20T13:56:34.323 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.320+0000 7f5b64d3b700 1 -- 172.21.15.40:0/4017889590 wait complete. 2020-11-20T13:56:34.330 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2020-11-20T13:56:34.331 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.4 2020-11-20T13:56:34.446 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.446+0000 7f562946a700 1 Processor -- start 2020-11-20T13:56:34.447 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.446+0000 7f562946a700 1 -- start start 2020-11-20T13:56:34.453 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.447+0000 7f562946a700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f56240b6c80 0x7f56240b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:34.454 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.447+0000 7f562946a700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f56240b1450 0x7f56240b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:34.454 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.447+0000 7f562946a700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56240b0d10 con 0x7f5624144bd0 2020-11-20T13:56:34.454 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.447+0000 7f562946a700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5624145060 con 0x7f56240b1450 2020-11-20T13:56:34.455 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.447+0000 7f562946a700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56240b0fb0 con 0x7f56240b6c80 2020-11-20T13:56:34.455 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.447+0000 7f56237fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f56240b1450 0x7f56240b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:34.455 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.447+0000 7f56227fc700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5624144bd0 0x7f56240b40f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45478/0 (socket says 172.21.15.40:45478) 2020-11-20T13:56:34.455 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.447+0000 7f56227fc700 1 -- 172.21.15.40:0/3716125783 learned_addr learned my addr 172.21.15.40:0/3716125783 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:34.455 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.447+0000 7f5622ffd700 1 --2- 172.21.15.40:0/3716125783 >> v2:172.21.15.116:3300/0 conn(0x7f56240b6c80 0x7f56240b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:34.456 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.447+0000 7f56237fe700 1 -- 172.21.15.40:0/3716125783 >> v2:172.21.15.116:3300/0 conn(0x7f56240b6c80 msgr2=0x7f56240b31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:34.456 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.447+0000 7f56237fe700 1 --2- 172.21.15.40:0/3716125783 >> v2:172.21.15.116:3300/0 conn(0x7f56240b6c80 0x7f56240b31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.456 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.447+0000 7f56237fe700 1 -- 172.21.15.40:0/3716125783 >> v1:172.21.15.40:6789/0 conn(0x7f5624144bd0 legacy=0x7f56240b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:34.456 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.447+0000 7f56237fe700 1 -- 172.21.15.40:0/3716125783 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56240b1da0 con 0x7f56240b1450 2020-11-20T13:56:34.456 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.447+0000 7f5621ffb700 1 -- 172.21.15.40:0/3716125783 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3547783127 0 0) 0x7f56240b0d10 con 0x7f5624144bd0 2020-11-20T13:56:34.457 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.447+0000 7f56237fe700 1 --2- 172.21.15.40:0/3716125783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56240b1450 0x7f56240b18b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f561c002f30 tx=0x7f561c006dc0).ready entity=mon.0 client_cookie=3627a49e70b36c2a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:34.457 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.447+0000 7f5621ffb700 1 -- 172.21.15.40:0/3716125783 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f561c0097f0 con 0x7f56240b1450 2020-11-20T13:56:34.458 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.448+0000 7f5621ffb700 1 -- 172.21.15.40:0/3716125783 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f561c009950 con 0x7f56240b1450 2020-11-20T13:56:34.458 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.448+0000 7f5621ffb700 1 -- 172.21.15.40:0/3716125783 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f561c009cb0 con 0x7f56240b1450 2020-11-20T13:56:34.458 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.448+0000 7f562946a700 1 -- 172.21.15.40:0/3716125783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56240b1450 msgr2=0x7f56240b18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:34.458 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.448+0000 7f562946a700 1 --2- 172.21.15.40:0/3716125783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56240b1450 0x7f56240b18b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f561c002f30 tx=0x7f561c006dc0).stop 2020-11-20T13:56:34.458 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.448+0000 7f562946a700 1 -- 172.21.15.40:0/3716125783 shutdown_connections 2020-11-20T13:56:34.459 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.448+0000 7f562946a700 1 --2- 172.21.15.40:0/3716125783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56240b1450 0x7f56240b18b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.459 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.448+0000 7f562946a700 1 --2- 172.21.15.40:0/3716125783 >> v2:172.21.15.116:3300/0 conn(0x7f56240b6c80 0x7f56240b31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.459 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.448+0000 7f562946a700 1 -- 172.21.15.40:0/3716125783 >> 172.21.15.40:0/3716125783 conn(0x7f56240ba1d0 msgr2=0x7f56240b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:34.459 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.448+0000 7f562946a700 1 -- 172.21.15.40:0/3716125783 shutdown_connections 2020-11-20T13:56:34.459 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.448+0000 7f562946a700 1 -- 172.21.15.40:0/3716125783 wait complete. 2020-11-20T13:56:34.460 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.449+0000 7f562946a700 1 Processor -- start 2020-11-20T13:56:34.460 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.449+0000 7f562946a700 1 -- start start 2020-11-20T13:56:34.460 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.450+0000 7f562946a700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f56240b1450 0x7f562412dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:34.460 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.450+0000 7f562946a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56240b6c80 0x7f5624147c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:34.460 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.450+0000 7f562946a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5624144bd0 0x7f56241569f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:34.461 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.450+0000 7f562946a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5624143d40 con 0x7f56240b6c80 2020-11-20T13:56:34.461 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.450+0000 7f562946a700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562412e4b0 con 0x7f5624144bd0 2020-11-20T13:56:34.461 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.450+0000 7f562946a700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f562412d340 con 0x7f56240b1450 2020-11-20T13:56:34.461 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.450+0000 7f56227fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56240b6c80 0x7f5624147c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:34.461 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.450+0000 7f56227fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56240b6c80 0x7f5624147c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54888/0 (socket says 172.21.15.40:54888) 2020-11-20T13:56:34.462 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.450+0000 7f56227fc700 1 -- 172.21.15.40:0/674673154 learned_addr learned my addr 172.21.15.40:0/674673154 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:34.462 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.450+0000 7f56237fe700 1 --2- 172.21.15.40:0/674673154 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5624144bd0 0x7f56241569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:34.462 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.450+0000 7f5622ffd700 1 --2- 172.21.15.40:0/674673154 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f56240b1450 0x7f562412dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:34.462 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.450+0000 7f56227fc700 1 -- 172.21.15.40:0/674673154 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f56240b1450 msgr2=0x7f562412dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:34.462 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.450+0000 7f56227fc700 1 --2- 172.21.15.40:0/674673154 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f56240b1450 0x7f562412dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.463 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.450+0000 7f56227fc700 1 -- 172.21.15.40:0/674673154 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5624144bd0 msgr2=0x7f56241569f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:34.463 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.450+0000 7f56227fc700 1 --2- 172.21.15.40:0/674673154 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5624144bd0 0x7f56241569f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.463 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.450+0000 7f56227fc700 1 -- 172.21.15.40:0/674673154 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f562412d5e0 con 0x7f56240b6c80 2020-11-20T13:56:34.463 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.450+0000 7f56237fe700 1 --2- 172.21.15.40:0/674673154 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5624144bd0 0x7f56241569f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:34.463 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.450+0000 7f56227fc700 1 --2- 172.21.15.40:0/674673154 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56240b6c80 0x7f5624147c10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f5614001210 tx=0x7f5614007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:34.464 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.450+0000 7f5613fff700 1 -- 172.21.15.40:0/674673154 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5614005b70 con 0x7f56240b6c80 2020-11-20T13:56:34.464 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.450+0000 7f5613fff700 1 -- 172.21.15.40:0/674673154 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5614005cd0 con 0x7f56240b6c80 2020-11-20T13:56:34.464 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.451+0000 7f562946a700 1 -- 172.21.15.40:0/674673154 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f562412d880 con 0x7f56240b6c80 2020-11-20T13:56:34.464 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.451+0000 7f5613fff700 1 -- 172.21.15.40:0/674673154 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5614007300 con 0x7f56240b6c80 2020-11-20T13:56:34.464 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.451+0000 7f562946a700 1 -- 172.21.15.40:0/674673154 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5624148100 con 0x7f56240b6c80 2020-11-20T13:56:34.465 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.452+0000 7f562946a700 1 -- 172.21.15.40:0/674673154 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5624057050 con 0x7f56240b6c80 2020-11-20T13:56:34.465 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.452+0000 7f5613fff700 1 -- 172.21.15.40:0/674673154 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f5614010030 con 0x7f56240b6c80 2020-11-20T13:56:34.465 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.452+0000 7f5613fff700 1 --2- 172.21.15.40:0/674673154 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f5600051320 0x7f56000537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:34.465 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.452+0000 7f5613fff700 1 -- 172.21.15.40:0/674673154 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f56140690e0 con 0x7f56240b6c80 2020-11-20T13:56:34.465 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.452+0000 7f5622ffd700 1 --2- 172.21.15.40:0/674673154 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f5600051320 0x7f56000537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:34.466 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.452+0000 7f5622ffd700 1 --2- 172.21.15.40:0/674673154 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f5600051320 0x7f56000537e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f561800d000 tx=0x7f5618008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:34.467 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.456+0000 7f5613fff700 1 -- 172.21.15.40:0/674673154 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f5614041300 con 0x7f56240b6c80 2020-11-20T13:56:34.616 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.615+0000 7f562946a700 1 -- 172.21.15.40:0/674673154 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f56240b3e00 con 0x7f56240b6c80 2020-11-20T13:56:34.616 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.616+0000 7f5613fff700 1 -- 172.21.15.40:0/674673154 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5614044920 con 0x7f56240b6c80 2020-11-20T13:56:34.617 INFO:teuthology.orchestra.run.smithi040.stdout:55834574853 2020-11-20T13:56:34.618 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.617+0000 7f562946a700 1 -- 172.21.15.40:0/674673154 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f5600051320 msgr2=0x7f56000537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:34.618 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.617+0000 7f562946a700 1 --2- 172.21.15.40:0/674673154 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f5600051320 0x7f56000537e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f561800d000 tx=0x7f5618008040).stop 2020-11-20T13:56:34.618 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.617+0000 7f562946a700 1 -- 172.21.15.40:0/674673154 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56240b6c80 msgr2=0x7f5624147c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:34.618 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.617+0000 7f562946a700 1 --2- 172.21.15.40:0/674673154 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56240b6c80 0x7f5624147c10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f5614001210 tx=0x7f5614007d70).stop 2020-11-20T13:56:34.619 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.617+0000 7f562946a700 1 -- 172.21.15.40:0/674673154 shutdown_connections 2020-11-20T13:56:34.619 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.617+0000 7f562946a700 1 --2- 172.21.15.40:0/674673154 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f5600051320 0x7f56000537e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.619 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.617+0000 7f562946a700 1 --2- 172.21.15.40:0/674673154 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f56240b1450 0x7f562412dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.619 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.617+0000 7f562946a700 1 --2- 172.21.15.40:0/674673154 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5624144bd0 0x7f56241569f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.619 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.617+0000 7f562946a700 1 --2- 172.21.15.40:0/674673154 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56240b6c80 0x7f5624147c10 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.620 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.617+0000 7f562946a700 1 -- 172.21.15.40:0/674673154 >> 172.21.15.40:0/674673154 conn(0x7f56240ba1d0 msgr2=0x7f56240b7650 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:34.620 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.617+0000 7f562946a700 1 -- 172.21.15.40:0/674673154 shutdown_connections 2020-11-20T13:56:34.620 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.617+0000 7f562946a700 1 -- 172.21.15.40:0/674673154 wait complete. 2020-11-20T13:56:34.628 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.4 2020-11-20T13:56:34.628 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.5 2020-11-20T13:56:34.744 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.743+0000 7f738ed9d700 1 Processor -- start 2020-11-20T13:56:34.744 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.744+0000 7f738ed9d700 1 -- start start 2020-11-20T13:56:34.746 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.744+0000 7f738ed9d700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f73880b6300 0x7f73880b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:34.746 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.744+0000 7f738ed9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f738812a270 0x7f73880b0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:34.746 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.744+0000 7f738ed9d700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7388144ef0 con 0x7f7388144a90 2020-11-20T13:56:34.747 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.744+0000 7f738ed9d700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73880b3f10 con 0x7f738812a270 2020-11-20T13:56:34.747 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.744+0000 7f738ed9d700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73880b3370 con 0x7f73880b6300 2020-11-20T13:56:34.747 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.744+0000 7f738e59c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f738812a270 0x7f73880b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:34.747 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.744+0000 7f738d59a700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7388144a90 0x7f7388129b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45490/0 (socket says 172.21.15.40:45490) 2020-11-20T13:56:34.748 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.745+0000 7f738d59a700 1 -- 172.21.15.40:0/2396370984 learned_addr learned my addr 172.21.15.40:0/2396370984 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:34.748 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.745+0000 7f738dd9b700 1 --2- 172.21.15.40:0/2396370984 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f73880b6300 0x7f73880b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:34.748 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.745+0000 7f738e59c700 1 -- 172.21.15.40:0/2396370984 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f73880b6300 msgr2=0x7f73880b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:34.748 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.745+0000 7f738e59c700 1 --2- 172.21.15.40:0/2396370984 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f73880b6300 0x7f73880b2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.749 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.745+0000 7f738e59c700 1 -- 172.21.15.40:0/2396370984 >> v1:172.21.15.40:6789/0 conn(0x7f7388144a90 legacy=0x7f7388129b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:34.749 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.745+0000 7f738e59c700 1 -- 172.21.15.40:0/2396370984 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73880b3610 con 0x7f738812a270 2020-11-20T13:56:34.749 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.745+0000 7f738cd99700 1 -- 172.21.15.40:0/2396370984 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 165964290 0 0) 0x7f73840010d0 con 0x7f7388144a90 2020-11-20T13:56:34.749 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.745+0000 7f738e59c700 1 --2- 172.21.15.40:0/2396370984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f738812a270 0x7f73880b0840 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f73800032b0 tx=0x7f73800091a0).ready entity=mon.0 client_cookie=99b791ac5e9dca95 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:34.749 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.745+0000 7f738cd99700 1 -- 172.21.15.40:0/2396370984 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f738000ba50 con 0x7f738812a270 2020-11-20T13:56:34.750 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.745+0000 7f738cd99700 1 -- 172.21.15.40:0/2396370984 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f738000bbb0 con 0x7f738812a270 2020-11-20T13:56:34.750 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.745+0000 7f738cd99700 1 -- 172.21.15.40:0/2396370984 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f738000bd20 con 0x7f738812a270 2020-11-20T13:56:34.750 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.745+0000 7f738ed9d700 1 -- 172.21.15.40:0/2396370984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f738812a270 msgr2=0x7f73880b0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:34.751 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.745+0000 7f738ed9d700 1 --2- 172.21.15.40:0/2396370984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f738812a270 0x7f73880b0840 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f73800032b0 tx=0x7f73800091a0).stop 2020-11-20T13:56:34.751 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.746+0000 7f738ed9d700 1 -- 172.21.15.40:0/2396370984 shutdown_connections 2020-11-20T13:56:34.751 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.746+0000 7f738ed9d700 1 --2- 172.21.15.40:0/2396370984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f738812a270 0x7f73880b0840 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.751 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.746+0000 7f738ed9d700 1 --2- 172.21.15.40:0/2396370984 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f73880b6300 0x7f73880b2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.752 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.746+0000 7f738ed9d700 1 -- 172.21.15.40:0/2396370984 >> 172.21.15.40:0/2396370984 conn(0x7f738805f390 msgr2=0x7f738805f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:34.752 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.746+0000 7f738ed9d700 1 -- 172.21.15.40:0/2396370984 shutdown_connections 2020-11-20T13:56:34.752 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.746+0000 7f738ed9d700 1 -- 172.21.15.40:0/2396370984 wait complete. 2020-11-20T13:56:34.755 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.747+0000 7f738ed9d700 1 Processor -- start 2020-11-20T13:56:34.755 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.747+0000 7f738ed9d700 1 -- start start 2020-11-20T13:56:34.755 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.747+0000 7f738ed9d700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f73880b6300 0x7f7388062ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:34.756 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.747+0000 7f738ed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f738812a270 0x7f7388067340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:34.756 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.747+0000 7f738ed9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7388144a90 0x7f738814e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:34.757 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.747+0000 7f738ed9d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7388143a20 con 0x7f738812a270 2020-11-20T13:56:34.757 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.747+0000 7f738ed9d700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73881580b0 con 0x7f7388144a90 2020-11-20T13:56:34.757 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.747+0000 7f738ed9d700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7388158350 con 0x7f73880b6300 2020-11-20T13:56:34.757 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.747+0000 7f738d59a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f738812a270 0x7f7388067340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:34.757 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.747+0000 7f738d59a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f738812a270 0x7f7388067340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54902/0 (socket says 172.21.15.40:54902) 2020-11-20T13:56:34.758 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.747+0000 7f738d59a700 1 -- 172.21.15.40:0/2523103374 learned_addr learned my addr 172.21.15.40:0/2523103374 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:34.758 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.748+0000 7f738dd9b700 1 --2- 172.21.15.40:0/2523103374 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f73880b6300 0x7f7388062ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:34.758 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.748+0000 7f738e59c700 1 --2- 172.21.15.40:0/2523103374 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7388144a90 0x7f738814e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:34.758 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.748+0000 7f738d59a700 1 -- 172.21.15.40:0/2523103374 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f73880b6300 msgr2=0x7f7388062ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:34.758 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.748+0000 7f738d59a700 1 --2- 172.21.15.40:0/2523103374 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f73880b6300 0x7f7388062ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.759 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.748+0000 7f738d59a700 1 -- 172.21.15.40:0/2523103374 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7388144a90 msgr2=0x7f738814e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:34.760 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.748+0000 7f738d59a700 1 --2- 172.21.15.40:0/2523103374 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7388144a90 0x7f738814e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.760 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.748+0000 7f738d59a700 1 -- 172.21.15.40:0/2523103374 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73881585f0 con 0x7f738812a270 2020-11-20T13:56:34.760 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.748+0000 7f738d59a700 1 --2- 172.21.15.40:0/2523103374 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f738812a270 0x7f7388067340 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f7384000b80 tx=0x7f73840078b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:34.760 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.748+0000 7f737affd700 1 -- 172.21.15.40:0/2523103374 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f73840073b0 con 0x7f738812a270 2020-11-20T13:56:34.760 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.748+0000 7f737affd700 1 -- 172.21.15.40:0/2523103374 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7384007510 con 0x7f738812a270 2020-11-20T13:56:34.762 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.748+0000 7f738ed9d700 1 -- 172.21.15.40:0/2523103374 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7388157cf0 con 0x7f738812a270 2020-11-20T13:56:34.762 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.748+0000 7f737affd700 1 -- 172.21.15.40:0/2523103374 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f73840057d0 con 0x7f738812a270 2020-11-20T13:56:34.762 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.748+0000 7f738ed9d700 1 -- 172.21.15.40:0/2523103374 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7388067830 con 0x7f738812a270 2020-11-20T13:56:34.762 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.749+0000 7f737affd700 1 -- 172.21.15.40:0/2523103374 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f738401a060 con 0x7f738812a270 2020-11-20T13:56:34.763 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.749+0000 7f738ed9d700 1 -- 172.21.15.40:0/2523103374 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7388157ee0 con 0x7f738812a270 2020-11-20T13:56:34.763 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.749+0000 7f737affd700 1 --2- 172.21.15.40:0/2523103374 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f7368051320 0x7f73680537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:34.763 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.750+0000 7f737affd700 1 -- 172.21.15.40:0/2523103374 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f7384068740 con 0x7f738812a270 2020-11-20T13:56:34.763 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.750+0000 7f738dd9b700 1 --2- 172.21.15.40:0/2523103374 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f7368051320 0x7f73680537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:34.763 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.750+0000 7f738dd9b700 1 --2- 172.21.15.40:0/2523103374 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f7368051320 0x7f73680537e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f737c00c000 tx=0x7f737c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:34.764 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.753+0000 7f737affd700 1 -- 172.21.15.40:0/2523103374 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f7384040960 con 0x7f738812a270 2020-11-20T13:56:34.915 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.915+0000 7f738ed9d700 1 -- 172.21.15.40:0/2523103374 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f7388144ef0 con 0x7f738812a270 2020-11-20T13:56:34.916 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.915+0000 7f737affd700 1 -- 172.21.15.40:0/2523103374 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7388144ef0 con 0x7f738812a270 2020-11-20T13:56:34.916 INFO:teuthology.orchestra.run.smithi040.stdout:55834574854 2020-11-20T13:56:34.917 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.917+0000 7f738ed9d700 1 -- 172.21.15.40:0/2523103374 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f7368051320 msgr2=0x7f73680537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:34.918 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.917+0000 7f738ed9d700 1 --2- 172.21.15.40:0/2523103374 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f7368051320 0x7f73680537e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f737c00c000 tx=0x7f737c008040).stop 2020-11-20T13:56:34.918 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.917+0000 7f738ed9d700 1 -- 172.21.15.40:0/2523103374 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f738812a270 msgr2=0x7f7388067340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:34.918 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.917+0000 7f738ed9d700 1 --2- 172.21.15.40:0/2523103374 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f738812a270 0x7f7388067340 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f7384000b80 tx=0x7f73840078b0).stop 2020-11-20T13:56:34.918 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.917+0000 7f738ed9d700 1 -- 172.21.15.40:0/2523103374 shutdown_connections 2020-11-20T13:56:34.918 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.917+0000 7f738ed9d700 1 --2- 172.21.15.40:0/2523103374 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f7368051320 0x7f73680537e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.919 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.917+0000 7f738ed9d700 1 --2- 172.21.15.40:0/2523103374 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f73880b6300 0x7f7388062ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.919 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.917+0000 7f738ed9d700 1 --2- 172.21.15.40:0/2523103374 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7388144a90 0x7f738814e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.919 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.917+0000 7f738ed9d700 1 --2- 172.21.15.40:0/2523103374 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f738812a270 0x7f7388067340 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.919 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.917+0000 7f738ed9d700 1 -- 172.21.15.40:0/2523103374 >> 172.21.15.40:0/2523103374 conn(0x7f738805f390 msgr2=0x7f73880b4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:34.920 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.917+0000 7f738ed9d700 1 -- 172.21.15.40:0/2523103374 shutdown_connections 2020-11-20T13:56:34.920 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:34.917+0000 7f738ed9d700 1 -- 172.21.15.40:0/2523103374 wait complete. 2020-11-20T13:56:34.927 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2020-11-20T13:56:34.928 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.6 2020-11-20T13:56:35.043 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.043+0000 7f8a09e17700 1 Processor -- start 2020-11-20T13:56:35.043 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.043+0000 7f8a09e17700 1 -- start start 2020-11-20T13:56:35.050 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.043+0000 7f8a09e17700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8a040b31f0 0x7f8a04144bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:35.050 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.043+0000 7f8a09e17700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8a040b1450 0x7f8a040b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:35.050 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.043+0000 7f8a09e17700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a040b1dd0 con 0x7f8a040b6c80 2020-11-20T13:56:35.050 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.043+0000 7f8a09e17700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a040b0d10 con 0x7f8a040b1450 2020-11-20T13:56:35.051 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.043+0000 7f8a09e17700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a040b0fb0 con 0x7f8a040b31f0 2020-11-20T13:56:35.051 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.044+0000 7f8a03fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8a040b1450 0x7f8a040b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:35.051 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.044+0000 7f8a03fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8a040b1450 0x7f8a040b18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54910/0 (socket says 172.21.15.40:54910) 2020-11-20T13:56:35.051 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.044+0000 7f8a03fff700 1 -- 172.21.15.40:0/1078523826 learned_addr learned my addr 172.21.15.40:0/1078523826 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:35.052 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.044+0000 7f8a02ffd700 1 --2- 172.21.15.40:0/1078523826 >> v2:172.21.15.116:3300/0 conn(0x7f8a040b31f0 0x7f8a04144bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:35.052 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.044+0000 7f8a03fff700 1 -- 172.21.15.40:0/1078523826 >> v2:172.21.15.116:3300/0 conn(0x7f8a040b31f0 msgr2=0x7f8a04144bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:35.052 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.044+0000 7f8a03fff700 1 --2- 172.21.15.40:0/1078523826 >> v2:172.21.15.116:3300/0 conn(0x7f8a040b31f0 0x7f8a04144bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.052 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.044+0000 7f8a03fff700 1 -- 172.21.15.40:0/1078523826 >> v1:172.21.15.116:6789/0 conn(0x7f8a040b6c80 legacy=0x7f8a040b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:35.053 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.044+0000 7f8a03fff700 1 -- 172.21.15.40:0/1078523826 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a040b7060 con 0x7f8a040b1450 2020-11-20T13:56:35.053 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.044+0000 7f8a03fff700 1 --2- 172.21.15.40:0/1078523826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a040b1450 0x7f8a040b18e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f8a040687d0 tx=0x7f89fc009270).ready entity=mon.0 client_cookie=947a623f0ba9070e server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:35.053 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.044+0000 7f8a027fc700 1 -- 172.21.15.40:0/1078523826 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89fc00bbc0 con 0x7f8a040b1450 2020-11-20T13:56:35.053 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.044+0000 7f8a027fc700 1 -- 172.21.15.40:0/1078523826 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89fc00bd20 con 0x7f8a040b1450 2020-11-20T13:56:35.054 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.044+0000 7f8a027fc700 1 -- 172.21.15.40:0/1078523826 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89fc011400 con 0x7f8a040b1450 2020-11-20T13:56:35.054 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.044+0000 7f8a09e17700 1 -- 172.21.15.40:0/1078523826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a040b1450 msgr2=0x7f8a040b18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:35.054 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.044+0000 7f8a09e17700 1 --2- 172.21.15.40:0/1078523826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a040b1450 0x7f8a040b18e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f8a040687d0 tx=0x7f89fc009270).stop 2020-11-20T13:56:35.054 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.045+0000 7f8a09e17700 1 -- 172.21.15.40:0/1078523826 shutdown_connections 2020-11-20T13:56:35.054 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.045+0000 7f8a09e17700 1 --2- 172.21.15.40:0/1078523826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a040b1450 0x7f8a040b18e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.055 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.045+0000 7f8a09e17700 1 --2- 172.21.15.40:0/1078523826 >> v2:172.21.15.116:3300/0 conn(0x7f8a040b31f0 0x7f8a04144bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.055 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.045+0000 7f8a09e17700 1 -- 172.21.15.40:0/1078523826 >> 172.21.15.40:0/1078523826 conn(0x7f8a040ba1d0 msgr2=0x7f8a040b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:35.055 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.045+0000 7f8a09e17700 1 -- 172.21.15.40:0/1078523826 shutdown_connections 2020-11-20T13:56:35.055 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.045+0000 7f8a09e17700 1 -- 172.21.15.40:0/1078523826 wait complete. 2020-11-20T13:56:35.055 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.046+0000 7f8a09e17700 1 Processor -- start 2020-11-20T13:56:35.056 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.046+0000 7f8a09e17700 1 -- start start 2020-11-20T13:56:35.056 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.046+0000 7f8a09e17700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a040b1450 0x7f8a04124390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:35.056 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.046+0000 7f8a09e17700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8a040b31f0 0x7f8a04065010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:35.057 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.046+0000 7f8a09e17700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a040b6c80 0x7f8a040accf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:35.057 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.046+0000 7f8a09e17700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a040629f0 con 0x7f8a040b6c80 2020-11-20T13:56:35.057 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.046+0000 7f8a09e17700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a04143be0 con 0x7f8a040b1450 2020-11-20T13:56:35.058 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.046+0000 7f8a09e17700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a04062c50 con 0x7f8a040b31f0 2020-11-20T13:56:35.058 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.047+0000 7f8a03fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a040b6c80 0x7f8a040accf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:35.058 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.047+0000 7f8a03fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a040b6c80 0x7f8a040accf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54918/0 (socket says 172.21.15.40:54918) 2020-11-20T13:56:35.058 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.047+0000 7f8a03fff700 1 -- 172.21.15.40:0/2661380911 learned_addr learned my addr 172.21.15.40:0/2661380911 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:35.058 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.047+0000 7f8a037fe700 1 --2- 172.21.15.40:0/2661380911 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a040b1450 0x7f8a04124390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:35.059 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.047+0000 7f8a02ffd700 1 --2- 172.21.15.40:0/2661380911 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8a040b31f0 0x7f8a04065010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:35.059 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.047+0000 7f8a03fff700 1 -- 172.21.15.40:0/2661380911 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8a040b31f0 msgr2=0x7f8a04065010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:35.059 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.047+0000 7f8a03fff700 1 --2- 172.21.15.40:0/2661380911 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8a040b31f0 0x7f8a04065010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.059 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.047+0000 7f8a03fff700 1 -- 172.21.15.40:0/2661380911 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a040b1450 msgr2=0x7f8a04124390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:35.059 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.047+0000 7f8a03fff700 1 --2- 172.21.15.40:0/2661380911 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a040b1450 0x7f8a04124390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.060 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.047+0000 7f8a03fff700 1 -- 172.21.15.40:0/2661380911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a04065500 con 0x7f8a040b6c80 2020-11-20T13:56:35.060 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.047+0000 7f8a037fe700 1 --2- 172.21.15.40:0/2661380911 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a040b1450 0x7f8a04124390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:35.060 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.047+0000 7f8a03fff700 1 --2- 172.21.15.40:0/2661380911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a040b6c80 0x7f8a040accf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f8a040687d0 tx=0x7f89fc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:35.060 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.047+0000 7f8a00ff9700 1 -- 172.21.15.40:0/2661380911 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89fc01f670 con 0x7f8a040b6c80 2020-11-20T13:56:35.060 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.047+0000 7f8a00ff9700 1 -- 172.21.15.40:0/2661380911 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89fc01f7d0 con 0x7f8a040b6c80 2020-11-20T13:56:35.061 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.047+0000 7f8a09e17700 1 -- 172.21.15.40:0/2661380911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a04065750 con 0x7f8a040b6c80 2020-11-20T13:56:35.061 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.047+0000 7f8a00ff9700 1 -- 172.21.15.40:0/2661380911 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89fc01fb30 con 0x7f8a040b6c80 2020-11-20T13:56:35.061 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.047+0000 7f8a09e17700 1 -- 172.21.15.40:0/2661380911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a040ad3a0 con 0x7f8a040b6c80 2020-11-20T13:56:35.061 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.048+0000 7f8a00ff9700 1 -- 172.21.15.40:0/2661380911 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f89fc026030 con 0x7f8a040b6c80 2020-11-20T13:56:35.062 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.048+0000 7f8a09e17700 1 -- 172.21.15.40:0/2661380911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a04057050 con 0x7f8a040b6c80 2020-11-20T13:56:35.062 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.048+0000 7f8a00ff9700 1 --2- 172.21.15.40:0/2661380911 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f89e4051290 0x7f89e4053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:35.062 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.048+0000 7f8a00ff9700 1 -- 172.21.15.40:0/2661380911 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f89fc06a680 con 0x7f8a040b6c80 2020-11-20T13:56:35.062 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.049+0000 7f8a037fe700 1 --2- 172.21.15.40:0/2661380911 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f89e4051290 0x7f89e4053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:35.062 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.049+0000 7f8a037fe700 1 --2- 172.21.15.40:0/2661380911 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f89e4051290 0x7f89e4053750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f89f400b000 tx=0x7f89f4009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:35.063 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.052+0000 7f8a00ff9700 1 -- 172.21.15.40:0/2661380911 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f89fc0462d0 con 0x7f8a040b6c80 2020-11-20T13:56:35.215 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.214+0000 7f8a09e17700 1 -- 172.21.15.40:0/2661380911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f8a040b9410 con 0x7f8a040b6c80 2020-11-20T13:56:35.215 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.215+0000 7f8a00ff9700 1 -- 172.21.15.40:0/2661380911 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f89fc024020 con 0x7f8a040b6c80 2020-11-20T13:56:35.215 INFO:teuthology.orchestra.run.smithi040.stdout:55834574853 2020-11-20T13:56:35.217 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.216+0000 7f8a09e17700 1 -- 172.21.15.40:0/2661380911 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f89e4051290 msgr2=0x7f89e4053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:35.217 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.216+0000 7f8a09e17700 1 --2- 172.21.15.40:0/2661380911 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f89e4051290 0x7f89e4053750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f89f400b000 tx=0x7f89f4009040).stop 2020-11-20T13:56:35.217 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.216+0000 7f8a09e17700 1 -- 172.21.15.40:0/2661380911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a040b6c80 msgr2=0x7f8a040accf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:35.218 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.216+0000 7f8a09e17700 1 --2- 172.21.15.40:0/2661380911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a040b6c80 0x7f8a040accf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f8a040687d0 tx=0x7f89fc016000).stop 2020-11-20T13:56:35.218 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.216+0000 7f8a09e17700 1 -- 172.21.15.40:0/2661380911 shutdown_connections 2020-11-20T13:56:35.218 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.216+0000 7f8a09e17700 1 --2- 172.21.15.40:0/2661380911 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f89e4051290 0x7f89e4053750 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.218 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.216+0000 7f8a09e17700 1 --2- 172.21.15.40:0/2661380911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a040b6c80 0x7f8a040accf0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.219 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.216+0000 7f8a09e17700 1 --2- 172.21.15.40:0/2661380911 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a040b1450 0x7f8a04124390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.219 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.216+0000 7f8a09e17700 1 --2- 172.21.15.40:0/2661380911 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8a040b31f0 0x7f8a04065010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.219 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.216+0000 7f8a09e17700 1 -- 172.21.15.40:0/2661380911 >> 172.21.15.40:0/2661380911 conn(0x7f8a040ba1d0 msgr2=0x7f8a040b8840 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:35.219 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.216+0000 7f8a09e17700 1 -- 172.21.15.40:0/2661380911 shutdown_connections 2020-11-20T13:56:35.219 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.216+0000 7f8a09e17700 1 -- 172.21.15.40:0/2661380911 wait complete. 2020-11-20T13:56:35.229 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.6 2020-11-20T13:56:35.229 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.7 2020-11-20T13:56:35.344 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.344+0000 7f641c2d6700 1 Processor -- start 2020-11-20T13:56:35.345 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.344+0000 7f641c2d6700 1 -- start start 2020-11-20T13:56:35.356 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.345+0000 7f641c2d6700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64140b1b60 0x7f6414144bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:35.356 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.345+0000 7f641c2d6700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64140b0d10 con 0x7f64140b31f0 2020-11-20T13:56:35.356 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.345+0000 7f641c2d6700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64140b0fb0 con 0x7f64140b6c80 2020-11-20T13:56:35.356 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.345+0000 7f641c2d6700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64140b7060 con 0x7f64140b1b60 2020-11-20T13:56:35.357 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.345+0000 7f6419871700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f64140b31f0 0x7f64140b1450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45518/0 (socket says 172.21.15.40:45518) 2020-11-20T13:56:35.357 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.345+0000 7f6419871700 1 -- 172.21.15.40:0/221253283 learned_addr learned my addr 172.21.15.40:0/221253283 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:35.357 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.345+0000 7f641a873700 1 --2- 172.21.15.40:0/221253283 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64140b1b60 0x7f6414144bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:35.357 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.345+0000 7f6419070700 1 -- 172.21.15.40:0/221253283 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1976075127 0 0) 0x7f64140b0d10 con 0x7f64140b31f0 2020-11-20T13:56:35.357 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.345+0000 7f6419070700 1 -- 172.21.15.40:0/221253283 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63fc003230 con 0x7f64140b31f0 2020-11-20T13:56:35.358 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.346+0000 7f6419070700 1 -- 172.21.15.40:0/221253283 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3931842649 0 0) 0x7f64140b0fb0 con 0x7f64140b6c80 2020-11-20T13:56:35.358 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.346+0000 7f6419070700 1 -- 172.21.15.40:0/221253283 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64140b0d10 con 0x7f64140b6c80 2020-11-20T13:56:35.358 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.346+0000 7f6419070700 1 -- 172.21.15.40:0/221253283 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1968398672 0 0) 0x7f63fc003230 con 0x7f64140b31f0 2020-11-20T13:56:35.358 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.346+0000 7f6419070700 1 -- 172.21.15.40:0/221253283 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64140b1b60 msgr2=0x7f6414144bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:35.358 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.346+0000 7f6419070700 1 --2- 172.21.15.40:0/221253283 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64140b1b60 0x7f6414144bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.359 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.346+0000 7f6419070700 1 -- 172.21.15.40:0/221253283 >> v1:172.21.15.116:6789/0 conn(0x7f64140b6c80 legacy=0x7f64140b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:35.359 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.346+0000 7f6419070700 1 -- 172.21.15.40:0/221253283 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64140b4800 con 0x7f64140b31f0 2020-11-20T13:56:35.359 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.346+0000 7f6419070700 1 -- 172.21.15.40:0/221253283 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2366654341 0 0) 0x7f64100016d0 con 0x7f64140b31f0 2020-11-20T13:56:35.359 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.346+0000 7f641a873700 1 --2- 172.21.15.40:0/221253283 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64140b1b60 0x7f6414144bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:56:35.359 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.346+0000 7f6419070700 1 -- 172.21.15.40:0/221253283 >> v1:172.21.15.40:6789/0 conn(0x7f64140b31f0 legacy=0x7f64140b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:35.360 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.346+0000 7f6419070700 1 --2- 172.21.15.40:0/221253283 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64140b1b60 0x7f63fc0038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:35.360 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.346+0000 7f6419070700 1 --2- 172.21.15.40:0/221253283 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63fc003dd0 0x7f63fc0044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:35.360 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.346+0000 7f6419070700 1 --2- 172.21.15.40:0/221253283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63fc0049c0 0x7f63fc004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:35.360 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.346+0000 7f6419070700 1 -- 172.21.15.40:0/221253283 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64140b7060 con 0x7f63fc0049c0 2020-11-20T13:56:35.360 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.346+0000 7f6419070700 1 -- 172.21.15.40:0/221253283 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63fc005360 con 0x7f63fc003dd0 2020-11-20T13:56:35.361 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.346+0000 7f6419070700 1 -- 172.21.15.40:0/221253283 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63fc00b770 con 0x7f64140b1b60 2020-11-20T13:56:35.361 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.346+0000 7f641a873700 1 --2- 172.21.15.40:0/221253283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63fc0049c0 0x7f63fc004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:35.361 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.346+0000 7f6419871700 1 --2- 172.21.15.40:0/221253283 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63fc003dd0 0x7f63fc0044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:35.361 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.346+0000 7f641a072700 1 --2- 172.21.15.40:0/221253283 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64140b1b60 0x7f63fc0038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:35.362 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.346+0000 7f641a873700 1 -- 172.21.15.40:0/221253283 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64140b1b60 msgr2=0x7f63fc0038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:35.362 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.346+0000 7f641a873700 1 --2- 172.21.15.40:0/221253283 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64140b1b60 0x7f63fc0038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.362 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.346+0000 7f641a873700 1 -- 172.21.15.40:0/221253283 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63fc003dd0 msgr2=0x7f63fc0044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:35.362 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.346+0000 7f641a873700 1 --2- 172.21.15.40:0/221253283 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63fc003dd0 0x7f63fc0044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.362 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.346+0000 7f641a873700 1 -- 172.21.15.40:0/221253283 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63fc003230 con 0x7f63fc0049c0 2020-11-20T13:56:35.363 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.346+0000 7f641a873700 1 --2- 172.21.15.40:0/221253283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63fc0049c0 0x7f63fc004e40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f63fc0054a0 tx=0x7f640c0091a0).ready entity=mon.0 client_cookie=e8ad909c132e9d0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:35.363 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.346+0000 7f6419070700 1 -- 172.21.15.40:0/221253283 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f640c00ed50 con 0x7f63fc0049c0 2020-11-20T13:56:35.363 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.346+0000 7f6419070700 1 -- 172.21.15.40:0/221253283 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f640c00eeb0 con 0x7f63fc0049c0 2020-11-20T13:56:35.363 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.346+0000 7f6419070700 1 -- 172.21.15.40:0/221253283 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f640c0125e0 con 0x7f63fc0049c0 2020-11-20T13:56:35.363 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.346+0000 7f641c2d6700 1 -- 172.21.15.40:0/221253283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63fc0049c0 msgr2=0x7f63fc004e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:35.364 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.346+0000 7f641c2d6700 1 --2- 172.21.15.40:0/221253283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63fc0049c0 0x7f63fc004e40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f63fc0054a0 tx=0x7f640c0091a0).stop 2020-11-20T13:56:35.364 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.346+0000 7f641a873700 1 -- 172.21.15.40:0/221253283 reap_dead start 2020-11-20T13:56:35.364 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.347+0000 7f641c2d6700 1 -- 172.21.15.40:0/221253283 shutdown_connections 2020-11-20T13:56:35.364 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.347+0000 7f641c2d6700 1 -- 172.21.15.40:0/221253283 >> 172.21.15.40:0/221253283 conn(0x7f64140ba1d0 msgr2=0x7f64140b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:35.365 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.347+0000 7f641c2d6700 1 -- 172.21.15.40:0/221253283 shutdown_connections 2020-11-20T13:56:35.365 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.347+0000 7f641c2d6700 1 -- 172.21.15.40:0/221253283 wait complete. 2020-11-20T13:56:35.365 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.348+0000 7f641c2d6700 1 Processor -- start 2020-11-20T13:56:35.366 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.348+0000 7f641c2d6700 1 -- start start 2020-11-20T13:56:35.368 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.349+0000 7f641c2d6700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6414129c20 0x7f64140644c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:35.368 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.349+0000 7f641c2d6700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6414155d20 0x7f6414065bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:35.369 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.349+0000 7f641c2d6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6414147cb0 0x7f6414148130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:35.369 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.349+0000 7f641c2d6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64140660c0 con 0x7f6414147cb0 2020-11-20T13:56:35.369 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.349+0000 7f641c2d6700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64141561a0 con 0x7f6414129c20 2020-11-20T13:56:35.369 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.349+0000 7f641c2d6700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f641412a000 con 0x7f6414155d20 2020-11-20T13:56:35.369 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.349+0000 7f641a873700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6414147cb0 0x7f6414148130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:35.370 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.349+0000 7f641a873700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6414147cb0 0x7f6414148130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54938/0 (socket says 172.21.15.40:54938) 2020-11-20T13:56:35.370 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.349+0000 7f641a873700 1 -- 172.21.15.40:0/3981099765 learned_addr learned my addr 172.21.15.40:0/3981099765 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:35.370 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.349+0000 7f6419871700 1 --2- 172.21.15.40:0/3981099765 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6414155d20 0x7f6414065bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:35.370 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.349+0000 7f641a072700 1 --2- 172.21.15.40:0/3981099765 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6414129c20 0x7f64140644c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:35.370 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.349+0000 7f641a873700 1 -- 172.21.15.40:0/3981099765 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6414155d20 msgr2=0x7f6414065bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:35.371 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.349+0000 7f641a873700 1 --2- 172.21.15.40:0/3981099765 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6414155d20 0x7f6414065bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.371 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.349+0000 7f641a873700 1 -- 172.21.15.40:0/3981099765 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6414129c20 msgr2=0x7f64140644c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:35.371 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.349+0000 7f641a873700 1 --2- 172.21.15.40:0/3981099765 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6414129c20 0x7f64140644c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.371 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.349+0000 7f641a873700 1 -- 172.21.15.40:0/3981099765 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6414066d90 con 0x7f6414147cb0 2020-11-20T13:56:35.372 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.349+0000 7f641a873700 1 --2- 172.21.15.40:0/3981099765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6414147cb0 0x7f6414148130 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f6414063f80 tx=0x7f640c01a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:35.372 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.349+0000 7f64037fe700 1 -- 172.21.15.40:0/3981099765 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f640c006c90 con 0x7f6414147cb0 2020-11-20T13:56:35.372 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.349+0000 7f64037fe700 1 -- 172.21.15.40:0/3981099765 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f640c0245e0 con 0x7f6414147cb0 2020-11-20T13:56:35.372 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.349+0000 7f641c2d6700 1 -- 172.21.15.40:0/3981099765 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64140afdc0 con 0x7f6414147cb0 2020-11-20T13:56:35.372 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.349+0000 7f64037fe700 1 -- 172.21.15.40:0/3981099765 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f640c024940 con 0x7f6414147cb0 2020-11-20T13:56:35.373 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.349+0000 7f641c2d6700 1 -- 172.21.15.40:0/3981099765 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6414151fb0 con 0x7f6414147cb0 2020-11-20T13:56:35.373 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.351+0000 7f641c2d6700 1 -- 172.21.15.40:0/3981099765 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64140aff50 con 0x7f6414147cb0 2020-11-20T13:56:35.373 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.351+0000 7f64037fe700 1 -- 172.21.15.40:0/3981099765 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f640c02c3f0 con 0x7f6414147cb0 2020-11-20T13:56:35.374 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.351+0000 7f64037fe700 1 --2- 172.21.15.40:0/3981099765 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f63f4051320 0x7f63f40537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:35.374 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.351+0000 7f64037fe700 1 -- 172.21.15.40:0/3981099765 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f640c06a840 con 0x7f6414147cb0 2020-11-20T13:56:35.374 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.351+0000 7f641a072700 1 --2- 172.21.15.40:0/3981099765 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f63f4051320 0x7f63f40537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:35.374 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.351+0000 7f641a072700 1 --2- 172.21.15.40:0/3981099765 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f63f4051320 0x7f63f40537e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f6404003000 tx=0x7f6404007b20).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:35.374 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.354+0000 7f64037fe700 1 -- 172.21.15.40:0/3981099765 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f640c042a00 con 0x7f6414147cb0 2020-11-20T13:56:35.518 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.517+0000 7f641c2d6700 1 -- 172.21.15.40:0/3981099765 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f6414057050 con 0x7f6414147cb0 2020-11-20T13:56:35.518 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.517+0000 7f64037fe700 1 -- 172.21.15.40:0/3981099765 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f640c016090 con 0x7f6414147cb0 2020-11-20T13:56:35.518 INFO:teuthology.orchestra.run.smithi040.stdout:55834574854 2020-11-20T13:56:35.519 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.519+0000 7f641c2d6700 1 -- 172.21.15.40:0/3981099765 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f63f4051320 msgr2=0x7f63f40537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:35.520 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.519+0000 7f641c2d6700 1 --2- 172.21.15.40:0/3981099765 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f63f4051320 0x7f63f40537e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f6404003000 tx=0x7f6404007b20).stop 2020-11-20T13:56:35.520 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.519+0000 7f641c2d6700 1 -- 172.21.15.40:0/3981099765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6414147cb0 msgr2=0x7f6414148130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:35.520 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.519+0000 7f641c2d6700 1 --2- 172.21.15.40:0/3981099765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6414147cb0 0x7f6414148130 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f6414063f80 tx=0x7f640c01a000).stop 2020-11-20T13:56:35.520 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.519+0000 7f641c2d6700 1 -- 172.21.15.40:0/3981099765 shutdown_connections 2020-11-20T13:56:35.521 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.519+0000 7f641c2d6700 1 --2- 172.21.15.40:0/3981099765 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f63f4051320 0x7f63f40537e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.521 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.519+0000 7f641c2d6700 1 --2- 172.21.15.40:0/3981099765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6414147cb0 0x7f6414148130 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.521 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.519+0000 7f641c2d6700 1 --2- 172.21.15.40:0/3981099765 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6414129c20 0x7f64140644c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.521 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.519+0000 7f641c2d6700 1 --2- 172.21.15.40:0/3981099765 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6414155d20 0x7f6414065bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.521 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.519+0000 7f641c2d6700 1 -- 172.21.15.40:0/3981099765 >> 172.21.15.40:0/3981099765 conn(0x7f64140ba1d0 msgr2=0x7f64140b7650 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:35.522 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.519+0000 7f641c2d6700 1 -- 172.21.15.40:0/3981099765 shutdown_connections 2020-11-20T13:56:35.522 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.519+0000 7f641c2d6700 1 -- 172.21.15.40:0/3981099765 wait complete. 2020-11-20T13:56:35.531 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2020-11-20T13:56:35.531 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2020-11-20T13:56:35.531 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-11-20T13:56:35.645 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.645+0000 7f84462ab700 1 Processor -- start 2020-11-20T13:56:35.646 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.645+0000 7f84462ab700 1 -- start start 2020-11-20T13:56:35.648 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.646+0000 7f84462ab700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f844009a280 0x7f84400b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:35.648 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.646+0000 7f84462ab700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f844009bc40 0x7f844009c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:35.648 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.646+0000 7f84462ab700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84400b6be0 con 0x7f84400b88c0 2020-11-20T13:56:35.649 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.646+0000 7f84462ab700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8440099000 con 0x7f844009bc40 2020-11-20T13:56:35.649 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.646+0000 7f84462ab700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84400b4250 con 0x7f844009a280 2020-11-20T13:56:35.649 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.646+0000 7f8444848700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f844009bc40 0x7f844009c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:35.649 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.646+0000 7f8444848700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f844009bc40 0x7f844009c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54946/0 (socket says 172.21.15.40:54946) 2020-11-20T13:56:35.649 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.646+0000 7f8444848700 1 -- 172.21.15.40:0/637340327 learned_addr learned my addr 172.21.15.40:0/637340327 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:35.650 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.646+0000 7f843f7fe700 1 --2- 172.21.15.40:0/637340327 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f844009a280 0x7f84400b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:35.650 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.646+0000 7f8444848700 1 -- 172.21.15.40:0/637340327 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f844009a280 msgr2=0x7f84400b8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:35.650 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.646+0000 7f8444848700 1 --2- 172.21.15.40:0/637340327 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f844009a280 0x7f84400b8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.651 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.646+0000 7f8444848700 1 -- 172.21.15.40:0/637340327 >> v1:172.21.15.116:6789/0 conn(0x7f84400b88c0 legacy=0x7f844009b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:35.651 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.646+0000 7f8444848700 1 -- 172.21.15.40:0/637340327 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84400b4390 con 0x7f844009bc40 2020-11-20T13:56:35.651 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.646+0000 7f8444848700 1 --2- 172.21.15.40:0/637340327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f844009bc40 0x7f844009c0a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f84400627d0 tx=0x7f8438009270).ready entity=mon.0 client_cookie=f8dcd5f6f1570fce server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:35.651 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.647+0000 7f843effd700 1 -- 172.21.15.40:0/637340327 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f843800bbc0 con 0x7f844009bc40 2020-11-20T13:56:35.652 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.647+0000 7f843effd700 1 -- 172.21.15.40:0/637340327 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f843800bd20 con 0x7f844009bc40 2020-11-20T13:56:35.652 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.647+0000 7f843effd700 1 -- 172.21.15.40:0/637340327 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8438011400 con 0x7f844009bc40 2020-11-20T13:56:35.652 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.647+0000 7f84462ab700 1 -- 172.21.15.40:0/637340327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f844009bc40 msgr2=0x7f844009c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:35.652 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.647+0000 7f84462ab700 1 --2- 172.21.15.40:0/637340327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f844009bc40 0x7f844009c0a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f84400627d0 tx=0x7f8438009270).stop 2020-11-20T13:56:35.653 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.647+0000 7f84462ab700 1 -- 172.21.15.40:0/637340327 shutdown_connections 2020-11-20T13:56:35.653 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.647+0000 7f84462ab700 1 --2- 172.21.15.40:0/637340327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f844009bc40 0x7f844009c0a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.653 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.647+0000 7f84462ab700 1 --2- 172.21.15.40:0/637340327 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f844009a280 0x7f84400b8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.653 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.647+0000 7f84462ab700 1 -- 172.21.15.40:0/637340327 >> 172.21.15.40:0/637340327 conn(0x7f84400ba380 msgr2=0x7f84400b6350 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:35.653 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.647+0000 7f84462ab700 1 -- 172.21.15.40:0/637340327 shutdown_connections 2020-11-20T13:56:35.654 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.647+0000 7f84462ab700 1 -- 172.21.15.40:0/637340327 wait complete. 2020-11-20T13:56:35.654 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.648+0000 7f84462ab700 1 Processor -- start 2020-11-20T13:56:35.654 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.648+0000 7f84462ab700 1 -- start start 2020-11-20T13:56:35.657 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.649+0000 7f84462ab700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f844009a280 0x7f844009d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:35.657 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.649+0000 7f84462ab700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f844009bc40 0x7f8440153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:35.657 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.649+0000 7f84462ab700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84400b88c0 0x7f84401471b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:35.657 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.649+0000 7f84462ab700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8440141720 con 0x7f844009bc40 2020-11-20T13:56:35.658 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.649+0000 7f84462ab700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84401578e0 con 0x7f84400b88c0 2020-11-20T13:56:35.658 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.649+0000 7f84462ab700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84400b1070 con 0x7f844009a280 2020-11-20T13:56:35.658 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.649+0000 7f843f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f844009bc40 0x7f8440153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:35.658 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.649+0000 7f843f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f844009bc40 0x7f8440153c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54950/0 (socket says 172.21.15.40:54950) 2020-11-20T13:56:35.658 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.649+0000 7f843f7fe700 1 -- 172.21.15.40:0/2808594325 learned_addr learned my addr 172.21.15.40:0/2808594325 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:35.659 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.649+0000 7f843ffff700 1 --2- 172.21.15.40:0/2808594325 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f844009a280 0x7f844009d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:35.659 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.649+0000 7f8444848700 1 --2- 172.21.15.40:0/2808594325 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84400b88c0 0x7f84401471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:35.659 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.649+0000 7f843f7fe700 1 -- 172.21.15.40:0/2808594325 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f844009a280 msgr2=0x7f844009d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:35.659 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.649+0000 7f843f7fe700 1 --2- 172.21.15.40:0/2808594325 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f844009a280 0x7f844009d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.659 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.649+0000 7f843f7fe700 1 -- 172.21.15.40:0/2808594325 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84400b88c0 msgr2=0x7f84401471b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:35.660 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.649+0000 7f843f7fe700 1 --2- 172.21.15.40:0/2808594325 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84400b88c0 0x7f84401471b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.660 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.649+0000 7f843f7fe700 1 -- 172.21.15.40:0/2808594325 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8440153500 con 0x7f844009bc40 2020-11-20T13:56:35.660 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.649+0000 7f843ffff700 1 --2- 172.21.15.40:0/2808594325 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f844009a280 0x7f844009d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:35.660 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.649+0000 7f843f7fe700 1 --2- 172.21.15.40:0/2808594325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f844009bc40 0x7f8440153c40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f844009dd20 tx=0x7f8434008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:35.661 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.650+0000 7f843d7fa700 1 -- 172.21.15.40:0/2808594325 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8434009980 con 0x7f844009bc40 2020-11-20T13:56:35.661 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.650+0000 7f843d7fa700 1 -- 172.21.15.40:0/2808594325 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8434009ae0 con 0x7f844009bc40 2020-11-20T13:56:35.661 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.650+0000 7f84462ab700 1 -- 172.21.15.40:0/2808594325 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84401537a0 con 0x7f844009bc40 2020-11-20T13:56:35.661 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.650+0000 7f843d7fa700 1 -- 172.21.15.40:0/2808594325 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8434009e40 con 0x7f844009bc40 2020-11-20T13:56:35.661 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.650+0000 7f84462ab700 1 -- 172.21.15.40:0/2808594325 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84400b2e40 con 0x7f844009bc40 2020-11-20T13:56:35.662 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.651+0000 7f84462ab700 1 -- 172.21.15.40:0/2808594325 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8440153930 con 0x7f844009bc40 2020-11-20T13:56:35.662 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.651+0000 7f843d7fa700 1 -- 172.21.15.40:0/2808594325 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f8434011450 con 0x7f844009bc40 2020-11-20T13:56:35.662 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.651+0000 7f843d7fa700 1 --2- 172.21.15.40:0/2808594325 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f8420051290 0x7f8420053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:35.662 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.651+0000 7f843d7fa700 1 -- 172.21.15.40:0/2808594325 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f843406b6f0 con 0x7f844009bc40 2020-11-20T13:56:35.662 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.651+0000 7f843ffff700 1 --2- 172.21.15.40:0/2808594325 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f8420051290 0x7f8420053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:35.663 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.651+0000 7f843ffff700 1 --2- 172.21.15.40:0/2808594325 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f8420051290 0x7f8420053750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f8430008000 tx=0x7f8430009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:35.663 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.654+0000 7f843d7fa700 1 -- 172.21.15.40:0/2808594325 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f84340438b0 con 0x7f844009bc40 2020-11-20T13:56:35.811 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.811+0000 7f84462ab700 1 -- 172.21.15.40:0/2808594325 --> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8440057050 con 0x7f8420051290 2020-11-20T13:56:35.820 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.815+0000 7f843d7fa700 1 -- 172.21.15.40:0/2808594325 <== mgr.4100 v2:172.21.15.116:6800/22233 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+82409 (secure 0 0 0) 0x7f8440057050 con 0x7f8420051290 2020-11-20T13:56:35.820 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.816+0000 7f84462ab700 1 -- 172.21.15.40:0/2808594325 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f8420051290 msgr2=0x7f8420053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:35.821 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.816+0000 7f84462ab700 1 --2- 172.21.15.40:0/2808594325 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f8420051290 0x7f8420053750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f8430008000 tx=0x7f8430009040).stop 2020-11-20T13:56:35.821 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.816+0000 7f84462ab700 1 -- 172.21.15.40:0/2808594325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f844009bc40 msgr2=0x7f8440153c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:35.821 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.816+0000 7f84462ab700 1 --2- 172.21.15.40:0/2808594325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f844009bc40 0x7f8440153c40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f844009dd20 tx=0x7f8434008040).stop 2020-11-20T13:56:35.821 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.816+0000 7f84462ab700 1 -- 172.21.15.40:0/2808594325 shutdown_connections 2020-11-20T13:56:35.822 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.816+0000 7f84462ab700 1 --2- 172.21.15.40:0/2808594325 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f8420051290 0x7f8420053750 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.822 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.816+0000 7f84462ab700 1 --2- 172.21.15.40:0/2808594325 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f844009a280 0x7f844009d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.822 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.816+0000 7f84462ab700 1 --2- 172.21.15.40:0/2808594325 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84400b88c0 0x7f84401471b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.822 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.816+0000 7f84462ab700 1 --2- 172.21.15.40:0/2808594325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f844009bc40 0x7f8440153c40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.823 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.816+0000 7f84462ab700 1 -- 172.21.15.40:0/2808594325 >> 172.21.15.40:0/2808594325 conn(0x7f84400ba380 msgr2=0x7f8440142ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:35.823 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.816+0000 7f84462ab700 1 -- 172.21.15.40:0/2808594325 shutdown_connections 2020-11-20T13:56:35.823 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.816+0000 7f84462ab700 1 -- 172.21.15.40:0/2808594325 wait complete. 2020-11-20T13:56:35.823 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2020-11-20T13:56:35.823 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:56:35.828 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2020-11-20T13:56:34.296512+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2304,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8390720,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746584000,"statfs":{"total":773094113280,"available":764502016000,"internally_reserved":8589934592,"allocated":2097152,"data_stored":826728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.258080"},"pg_stats":[{"pgid":"4.2","version":"0'0","reported_seq":"11","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-20T13:56:25.053797+0000","last_change":"2020-11-20T13:56:25.052346+0000","last_active":"2020-11-20T13:56:25.053797+0000","last_peered":"2020-11-20T13:56:25.053797+0000","last_clean":"2020-11-20T13:56:25.053797+0000","last_became_active":"2020-11-20T13:56:25.051763+0000","last_became_peered":"2020-11-20T13:56:25.051763+0000","last_unstale":"2020-11-20T13:56:25.053797+0000","last_undegraded":"2020-11-20T13:56:25.053797+0000","last_fullsized":"2020-11-20T13:56:25.053797+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:24.028995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:24.028995+0000","last_clean_scrub_stamp":"2020-11-20T13:56:24.028995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"15","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-20T13:56:25.044177+0000","last_change":"2020-11-20T13:56:19.143670+0000","last_active":"2020-11-20T13:56:25.044177+0000","last_peered":"2020-11-20T13:56:25.044177+0000","last_clean":"2020-11-20T13:56:25.044177+0000","last_became_active":"2020-11-20T13:56:19.143154+0000","last_became_peered":"2020-11-20T13:56:19.143154+0000","last_unstale":"2020-11-20T13:56:25.044177+0000","last_undegraded":"2020-11-20T13:56:25.044177+0000","last_fullsized":"2020-11-20T13:56:25.044177+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:17.989825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:17.989825+0000","last_clean_scrub_stamp":"2020-11-20T13:56:17.989825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"24'6","reported_seq":"18","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:56:25.450277+0000","last_change":"2020-11-20T13:56:22.036494+0000","last_active":"2020-11-20T13:56:25.450277+0000","last_peered":"2020-11-20T13:56:25.450277+0000","last_clean":"2020-11-20T13:56:25.450277+0000","last_became_active":"2020-11-20T13:56:22.036111+0000","last_became_peered":"2020-11-20T13:56:22.036111+0000","last_unstale":"2020-11-20T13:56:25.450277+0000","last_undegraded":"2020-11-20T13:56:25.450277+0000","last_fullsized":"2020-11-20T13:56:25.450277+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:21.011224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:21.011224+0000","last_clean_scrub_stamp":"2020-11-20T13:56:21.011224+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":"11","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-20T13:56:25.096170+0000","last_change":"2020-11-20T13:56:25.095407+0000","last_active":"2020-11-20T13:56:25.096170+0000","last_peered":"2020-11-20T13:56:25.096170+0000","last_clean":"2020-11-20T13:56:25.096170+0000","last_became_active":"2020-11-20T13:56:25.094738+0000","last_became_peered":"2020-11-20T13:56:25.094738+0000","last_unstale":"2020-11-20T13:56:25.096170+0000","last_undegraded":"2020-11-20T13:56:25.096170+0000","last_fullsized":"2020-11-20T13:56:25.096170+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:24.028995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:24.028995+0000","last_clean_scrub_stamp":"2020-11-20T13:56:24.028995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"15","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-20T13:56:25.040550+0000","last_change":"2020-11-20T13:56:19.010697+0000","last_active":"2020-11-20T13:56:25.040550+0000","last_peered":"2020-11-20T13:56:25.040550+0000","last_clean":"2020-11-20T13:56:25.040550+0000","last_became_active":"2020-11-20T13:56:19.010404+0000","last_became_peered":"2020-11-20T13:56:19.010404+0000","last_unstale":"2020-11-20T13:56:25.040550+0000","last_undegraded":"2020-11-20T13:56:25.040550+0000","last_fullsized":"2020-11-20T13:56:25.040550+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:17.989825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:17.989825+0000","last_clean_scrub_stamp":"2020-11-20T13:56:17.989825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"24'5","reported_seq":"17","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:56:25.447431+0000","last_change":"2020-11-20T13:56:22.038645+0000","last_active":"2020-11-20T13:56:25.447431+0000","last_peered":"2020-11-20T13:56:25.447431+0000","last_clean":"2020-11-20T13:56:25.447431+0000","last_became_active":"2020-11-20T13:56:22.038261+0000","last_became_peered":"2020-11-20T13:56:22.038261+0000","last_unstale":"2020-11-20T13:56:25.447431+0000","last_undegraded":"2020-11-20T13:56:25.447431+0000","last_fullsized":"2020-11-20T13:56:25.447431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:21.011224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:21.011224+0000","last_clean_scrub_stamp":"2020-11-20T13:56:21.011224+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":"11","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-20T13:56:25.051161+0000","last_change":"2020-11-20T13:56:25.049944+0000","last_active":"2020-11-20T13:56:25.051161+0000","last_peered":"2020-11-20T13:56:25.051161+0000","last_clean":"2020-11-20T13:56:25.051161+0000","last_became_active":"2020-11-20T13:56:25.049257+0000","last_became_peered":"2020-11-20T13:56:25.049257+0000","last_unstale":"2020-11-20T13:56:25.051161+0000","last_undegraded":"2020-11-20T13:56:25.051161+0000","last_fullsized":"2020-11-20T13:56:25.051161+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:24.028995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:24.028995+0000","last_clean_scrub_stamp":"2020-11-20T13:56:24.028995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"15","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-20T13:56:25.093139+0000","last_change":"2020-11-20T13:56:19.010270+0000","last_active":"2020-11-20T13:56:25.093139+0000","last_peered":"2020-11-20T13:56:25.093139+0000","last_clean":"2020-11-20T13:56:25.093139+0000","last_became_active":"2020-11-20T13:56:19.009804+0000","last_became_peered":"2020-11-20T13:56:19.009804+0000","last_unstale":"2020-11-20T13:56:25.093139+0000","last_undegraded":"2020-11-20T13:56:25.093139+0000","last_fullsized":"2020-11-20T13:56:25.093139+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:17.989825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:17.989825+0000","last_clean_scrub_stamp":"2020-11-20T13:56:17.989825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"24'8","reported_seq":"20","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:56:25.444883+0000","last_change":"2020-11-20T13:56:22.039123+0000","last_active":"2020-11-20T13:56:25.444883+0000","last_peered":"2020-11-20T13:56:25.444883+0000","last_clean":"2020-11-20T13:56:25.444883+0000","last_became_active":"2020-11-20T13:56:22.038692+0000","last_became_peered":"2020-11-20T13:56:22.038692+0000","last_unstale":"2020-11-20T13:56:25.444883+0000","last_undegraded":"2020-11-20T13:56:25.444883+0000","last_fullsized":"2020-11-20T13:56:25.444883+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:21.011224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:21.011224+0000","last_clean_scrub_stamp":"2020-11-20T13:56:21.011224+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":"11","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-20T13:56:25.056607+0000","last_change":"2020-11-20T13:56:25.050749+0000","last_active":"2020-11-20T13:56:25.056607+0000","last_peered":"2020-11-20T13:56:25.056607+0000","last_clean":"2020-11-20T13:56:25.056607+0000","last_became_active":"2020-11-20T13:56:25.049653+0000","last_became_peered":"2020-11-20T13:56:25.049653+0000","last_unstale":"2020-11-20T13:56:25.056607+0000","last_undegraded":"2020-11-20T13:56:25.056607+0000","last_fullsized":"2020-11-20T13:56:25.056607+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:24.028995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:24.028995+0000","last_clean_scrub_stamp":"2020-11-20T13:56:24.028995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"15","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-20T13:56:25.040571+0000","last_change":"2020-11-20T13:56:19.012731+0000","last_active":"2020-11-20T13:56:25.040571+0000","last_peered":"2020-11-20T13:56:25.040571+0000","last_clean":"2020-11-20T13:56:25.040571+0000","last_became_active":"2020-11-20T13:56:19.012477+0000","last_became_peered":"2020-11-20T13:56:19.012477+0000","last_unstale":"2020-11-20T13:56:25.040571+0000","last_undegraded":"2020-11-20T13:56:25.040571+0000","last_fullsized":"2020-11-20T13:56:25.040571+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:17.989825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:17.989825+0000","last_clean_scrub_stamp":"2020-11-20T13:56:17.989825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"20","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-20T13:56:25.040746+0000","last_change":"2020-11-20T13:56:15.047948+0000","last_active":"2020-11-20T13:56:25.040746+0000","last_peered":"2020-11-20T13:56:25.040746+0000","last_clean":"2020-11-20T13:56:25.040746+0000","last_became_active":"2020-11-20T13:56:15.047659+0000","last_became_peered":"2020-11-20T13:56:15.047659+0000","last_unstale":"2020-11-20T13:56:25.040746+0000","last_undegraded":"2020-11-20T13:56:25.040746+0000","last_fullsized":"2020-11-20T13:56:25.040746+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":6,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:08.473667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:08.473667+0000","last_clean_scrub_stamp":"2020-11-20T13:56:08.473667+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"24'1","reported_seq":"13","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:56:25.443619+0000","last_change":"2020-11-20T13:56:22.036039+0000","last_active":"2020-11-20T13:56:25.443619+0000","last_peered":"2020-11-20T13:56:25.443619+0000","last_clean":"2020-11-20T13:56:25.443619+0000","last_became_active":"2020-11-20T13:56:22.035637+0000","last_became_peered":"2020-11-20T13:56:22.035637+0000","last_unstale":"2020-11-20T13:56:25.443619+0000","last_undegraded":"2020-11-20T13:56:25.443619+0000","last_fullsized":"2020-11-20T13:56:25.443619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:21.011224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:21.011224+0000","last_clean_scrub_stamp":"2020-11-20T13:56:21.011224+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":"11","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-20T13:56:25.097051+0000","last_change":"2020-11-20T13:56:25.095551+0000","last_active":"2020-11-20T13:56:25.097051+0000","last_peered":"2020-11-20T13:56:25.097051+0000","last_clean":"2020-11-20T13:56:25.097051+0000","last_became_active":"2020-11-20T13:56:25.094843+0000","last_became_peered":"2020-11-20T13:56:25.094843+0000","last_unstale":"2020-11-20T13:56:25.097051+0000","last_undegraded":"2020-11-20T13:56:25.097051+0000","last_fullsized":"2020-11-20T13:56:25.097051+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:24.028995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:24.028995+0000","last_clean_scrub_stamp":"2020-11-20T13:56:24.028995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"15","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-20T13:56:25.040820+0000","last_change":"2020-11-20T13:56:19.008877+0000","last_active":"2020-11-20T13:56:25.040820+0000","last_peered":"2020-11-20T13:56:25.040820+0000","last_clean":"2020-11-20T13:56:25.040820+0000","last_became_active":"2020-11-20T13:56:19.008558+0000","last_became_peered":"2020-11-20T13:56:19.008558+0000","last_unstale":"2020-11-20T13:56:25.040820+0000","last_undegraded":"2020-11-20T13:56:25.040820+0000","last_fullsized":"2020-11-20T13:56:25.040820+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:17.989825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:17.989825+0000","last_clean_scrub_stamp":"2020-11-20T13:56:17.989825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"12","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-20T13:56:25.041657+0000","last_change":"2020-11-20T13:56:22.031448+0000","last_active":"2020-11-20T13:56:25.041657+0000","last_peered":"2020-11-20T13:56:25.041657+0000","last_clean":"2020-11-20T13:56:25.041657+0000","last_became_active":"2020-11-20T13:56:22.031157+0000","last_became_peered":"2020-11-20T13:56:22.031157+0000","last_unstale":"2020-11-20T13:56:25.041657+0000","last_undegraded":"2020-11-20T13:56:25.041657+0000","last_fullsized":"2020-11-20T13:56:25.041657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:21.011224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:21.011224+0000","last_clean_scrub_stamp":"2020-11-20T13:56:21.011224+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"11","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-20T13:56:25.050669+0000","last_change":"2020-11-20T13:56:25.049467+0000","last_active":"2020-11-20T13:56:25.050669+0000","last_peered":"2020-11-20T13:56:25.050669+0000","last_clean":"2020-11-20T13:56:25.050669+0000","last_became_active":"2020-11-20T13:56:25.048869+0000","last_became_peered":"2020-11-20T13:56:25.048869+0000","last_unstale":"2020-11-20T13:56:25.050669+0000","last_undegraded":"2020-11-20T13:56:25.050669+0000","last_fullsized":"2020-11-20T13:56:25.050669+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:24.028995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:24.028995+0000","last_clean_scrub_stamp":"2020-11-20T13:56:24.028995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"15","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-20T13:56:25.040938+0000","last_change":"2020-11-20T13:56:19.144378+0000","last_active":"2020-11-20T13:56:25.040938+0000","last_peered":"2020-11-20T13:56:25.040938+0000","last_clean":"2020-11-20T13:56:25.040938+0000","last_became_active":"2020-11-20T13:56:19.144086+0000","last_became_peered":"2020-11-20T13:56:19.144086+0000","last_unstale":"2020-11-20T13:56:25.040938+0000","last_undegraded":"2020-11-20T13:56:25.040938+0000","last_fullsized":"2020-11-20T13:56:25.040938+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:17.989825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:17.989825+0000","last_clean_scrub_stamp":"2020-11-20T13:56:17.989825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"24'8","reported_seq":"20","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:56:25.455025+0000","last_change":"2020-11-20T13:56:22.039342+0000","last_active":"2020-11-20T13:56:25.455025+0000","last_peered":"2020-11-20T13:56:25.455025+0000","last_clean":"2020-11-20T13:56:25.455025+0000","last_became_active":"2020-11-20T13:56:22.038953+0000","last_became_peered":"2020-11-20T13:56:22.038953+0000","last_unstale":"2020-11-20T13:56:25.455025+0000","last_undegraded":"2020-11-20T13:56:25.455025+0000","last_fullsized":"2020-11-20T13:56:25.455025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:21.011224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:21.011224+0000","last_clean_scrub_stamp":"2020-11-20T13:56:21.011224+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1535,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"11","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-20T13:56:25.051029+0000","last_change":"2020-11-20T13:56:25.050098+0000","last_active":"2020-11-20T13:56:25.051029+0000","last_peered":"2020-11-20T13:56:25.051029+0000","last_clean":"2020-11-20T13:56:25.051029+0000","last_became_active":"2020-11-20T13:56:25.049451+0000","last_became_peered":"2020-11-20T13:56:25.049451+0000","last_unstale":"2020-11-20T13:56:25.051029+0000","last_undegraded":"2020-11-20T13:56:25.051029+0000","last_fullsized":"2020-11-20T13:56:25.051029+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:24.028995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:24.028995+0000","last_clean_scrub_stamp":"2020-11-20T13:56:24.028995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"24'7","reported_seq":"19","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:56:25.450024+0000","last_change":"2020-11-20T13:56:22.033990+0000","last_active":"2020-11-20T13:56:25.450024+0000","last_peered":"2020-11-20T13:56:25.450024+0000","last_clean":"2020-11-20T13:56:25.450024+0000","last_became_active":"2020-11-20T13:56:22.033658+0000","last_became_peered":"2020-11-20T13:56:22.033658+0000","last_unstale":"2020-11-20T13:56:25.450024+0000","last_undegraded":"2020-11-20T13:56:25.450024+0000","last_fullsized":"2020-11-20T13:56:25.450024+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:21.011224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:21.011224+0000","last_clean_scrub_stamp":"2020-11-20T13:56:21.011224+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":575,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"15","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-20T13:56:25.044942+0000","last_change":"2020-11-20T13:56:19.013591+0000","last_active":"2020-11-20T13:56:25.044942+0000","last_peered":"2020-11-20T13:56:25.044942+0000","last_clean":"2020-11-20T13:56:25.044942+0000","last_became_active":"2020-11-20T13:56:19.013128+0000","last_became_peered":"2020-11-20T13:56:19.013128+0000","last_unstale":"2020-11-20T13:56:25.044942+0000","last_undegraded":"2020-11-20T13:56:25.044942+0000","last_fullsized":"2020-11-20T13:56:25.044942+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:17.989825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:17.989825+0000","last_clean_scrub_stamp":"2020-11-20T13:56:17.989825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"11","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-20T13:56:25.049208+0000","last_change":"2020-11-20T13:56:25.047910+0000","last_active":"2020-11-20T13:56:25.049208+0000","last_peered":"2020-11-20T13:56:25.049208+0000","last_clean":"2020-11-20T13:56:25.049208+0000","last_became_active":"2020-11-20T13:56:25.047294+0000","last_became_peered":"2020-11-20T13:56:25.047294+0000","last_unstale":"2020-11-20T13:56:25.049208+0000","last_undegraded":"2020-11-20T13:56:25.049208+0000","last_fullsized":"2020-11-20T13:56:25.049208+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:24.028995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:24.028995+0000","last_clean_scrub_stamp":"2020-11-20T13:56:24.028995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"14","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-20T13:56:25.044582+0000","last_change":"2020-11-20T13:56:19.015762+0000","last_active":"2020-11-20T13:56:25.044582+0000","last_peered":"2020-11-20T13:56:25.044582+0000","last_clean":"2020-11-20T13:56:25.044582+0000","last_became_active":"2020-11-20T13:56:19.015422+0000","last_became_peered":"2020-11-20T13:56:19.015422+0000","last_unstale":"2020-11-20T13:56:25.044582+0000","last_undegraded":"2020-11-20T13:56:25.044582+0000","last_fullsized":"2020-11-20T13:56:25.044582+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:17.989825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:17.989825+0000","last_clean_scrub_stamp":"2020-11-20T13:56:17.989825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"24'4","reported_seq":"16","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:56:25.445741+0000","last_change":"2020-11-20T13:56:22.039130+0000","last_active":"2020-11-20T13:56:25.445741+0000","last_peered":"2020-11-20T13:56:25.445741+0000","last_clean":"2020-11-20T13:56:25.445741+0000","last_became_active":"2020-11-20T13:56:22.038719+0000","last_became_peered":"2020-11-20T13:56:22.038719+0000","last_unstale":"2020-11-20T13:56:25.445741+0000","last_undegraded":"2020-11-20T13:56:25.445741+0000","last_fullsized":"2020-11-20T13:56:25.445741+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:21.011224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:21.011224+0000","last_clean_scrub_stamp":"2020-11-20T13:56:21.011224+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":32,"acting":32,"num_store_stats":8},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048840,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323000,"statfs":{"total":96636764160,"available":95562752000,"internally_reserved":1073741824,"allocated":262144,"data_stored":103340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1419999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3999999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048832,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323008,"statfs":{"total":96636764160,"available":95562760192,"internally_reserved":1073741824,"allocated":253952,"data_stored":102765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0860000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.319}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3779999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048844,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322996,"statfs":{"total":96636764160,"available":95562747904,"internally_reserved":1073741824,"allocated":266240,"data_stored":104300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.103}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2769999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2529999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048836,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323004,"statfs":{"total":96636764160,"available":95562756096,"internally_reserved":1073741824,"allocated":258048,"data_stored":102799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2150000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.149}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1739999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048844,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322996,"statfs":{"total":96636764160,"available":95562747904,"internally_reserved":1073741824,"allocated":266240,"data_stored":104300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2130000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048848,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322992,"statfs":{"total":96636764160,"available":95562743808,"internally_reserved":1073741824,"allocated":270336,"data_stored":103500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.452}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1160000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2290000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048836,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323004,"statfs":{"total":96636764160,"available":95562756096,"internally_reserved":1073741824,"allocated":258048,"data_stored":102799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.198}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.087}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.492}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048840,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323000,"statfs":{"total":96636764160,"available":95562752000,"internally_reserved":1073741824,"allocated":262144,"data_stored":102925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1479999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3160000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.246}]}]}]}} 2020-11-20T13:56:35.830 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-11-20T13:56:35.945 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.945+0000 7f6c75d58700 1 Processor -- start 2020-11-20T13:56:35.946 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.945+0000 7f6c75d58700 1 -- start start 2020-11-20T13:56:35.948 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.946+0000 7f6c75d58700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6c7009a280 0x7f6c700b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:35.948 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.946+0000 7f6c75d58700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6c7009bc40 0x7f6c7009c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:35.948 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.946+0000 7f6c75d58700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c700b6be0 con 0x7f6c700b88c0 2020-11-20T13:56:35.948 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.946+0000 7f6c75d58700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c70099000 con 0x7f6c7009bc40 2020-11-20T13:56:35.949 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.946+0000 7f6c75d58700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c700b4250 con 0x7f6c7009a280 2020-11-20T13:56:35.949 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.946+0000 7f6c6ffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6c7009bc40 0x7f6c7009c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:35.950 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.946+0000 7f6c6ffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6c7009bc40 0x7f6c7009c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54958/0 (socket says 172.21.15.40:54958) 2020-11-20T13:56:35.950 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.946+0000 7f6c6ffff700 1 -- 172.21.15.40:0/2848449771 learned_addr learned my addr 172.21.15.40:0/2848449771 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:35.950 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.946+0000 7f6c6effd700 1 --2- 172.21.15.40:0/2848449771 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6c7009a280 0x7f6c700b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:35.950 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.946+0000 7f6c6ffff700 1 -- 172.21.15.40:0/2848449771 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6c7009a280 msgr2=0x7f6c700b8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:35.951 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.946+0000 7f6c6ffff700 1 --2- 172.21.15.40:0/2848449771 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6c7009a280 0x7f6c700b8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.951 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.946+0000 7f6c6ffff700 1 -- 172.21.15.40:0/2848449771 >> v1:172.21.15.116:6789/0 conn(0x7f6c700b88c0 legacy=0x7f6c7009b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:35.951 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.946+0000 7f6c6ffff700 1 -- 172.21.15.40:0/2848449771 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c700b4390 con 0x7f6c7009bc40 2020-11-20T13:56:35.951 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.946+0000 7f6c6ffff700 1 --2- 172.21.15.40:0/2848449771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c7009bc40 0x7f6c7009c0a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f6c680032b0 tx=0x7f6c68009270).ready entity=mon.0 client_cookie=79acc6f615c32a29 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:35.951 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.947+0000 7f6c6e7fc700 1 -- 172.21.15.40:0/2848449771 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c68009e60 con 0x7f6c7009bc40 2020-11-20T13:56:35.952 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.947+0000 7f6c6e7fc700 1 -- 172.21.15.40:0/2848449771 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c6800bc00 con 0x7f6c7009bc40 2020-11-20T13:56:35.952 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.947+0000 7f6c6e7fc700 1 -- 172.21.15.40:0/2848449771 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c6800bd70 con 0x7f6c7009bc40 2020-11-20T13:56:35.952 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.947+0000 7f6c75d58700 1 -- 172.21.15.40:0/2848449771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c7009bc40 msgr2=0x7f6c7009c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:35.952 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.947+0000 7f6c75d58700 1 --2- 172.21.15.40:0/2848449771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c7009bc40 0x7f6c7009c0a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f6c680032b0 tx=0x7f6c68009270).stop 2020-11-20T13:56:35.953 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.947+0000 7f6c75d58700 1 -- 172.21.15.40:0/2848449771 shutdown_connections 2020-11-20T13:56:35.953 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.947+0000 7f6c75d58700 1 --2- 172.21.15.40:0/2848449771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c7009bc40 0x7f6c7009c0a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.953 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.947+0000 7f6c75d58700 1 --2- 172.21.15.40:0/2848449771 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6c7009a280 0x7f6c700b8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.953 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.947+0000 7f6c75d58700 1 -- 172.21.15.40:0/2848449771 >> 172.21.15.40:0/2848449771 conn(0x7f6c700ba380 msgr2=0x7f6c700b6350 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:35.953 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.947+0000 7f6c75d58700 1 -- 172.21.15.40:0/2848449771 shutdown_connections 2020-11-20T13:56:35.954 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.947+0000 7f6c75d58700 1 -- 172.21.15.40:0/2848449771 wait complete. 2020-11-20T13:56:35.956 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.948+0000 7f6c75d58700 1 Processor -- start 2020-11-20T13:56:35.957 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.948+0000 7f6c75d58700 1 -- start start 2020-11-20T13:56:35.957 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.949+0000 7f6c75d58700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6c7009a280 0x7f6c70146e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:35.957 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.949+0000 7f6c75d58700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6c7009bc40 0x7f6c700b3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:35.958 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.949+0000 7f6c75d58700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c700b88c0 0x7f6c70141e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:35.958 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.949+0000 7f6c75d58700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7009f700 con 0x7f6c700b88c0 2020-11-20T13:56:35.959 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.949+0000 7f6c75d58700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c70147b90 con 0x7f6c7009a280 2020-11-20T13:56:35.959 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.949+0000 7f6c75d58700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7009f260 con 0x7f6c7009bc40 2020-11-20T13:56:35.959 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.949+0000 7f6c6ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c700b88c0 0x7f6c70141e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:35.960 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.949+0000 7f6c6ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c700b88c0 0x7f6c70141e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54964/0 (socket says 172.21.15.40:54964) 2020-11-20T13:56:35.960 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.949+0000 7f6c6ffff700 1 -- 172.21.15.40:0/3387154340 learned_addr learned my addr 172.21.15.40:0/3387154340 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:35.960 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.949+0000 7f6c6f7fe700 1 --2- 172.21.15.40:0/3387154340 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6c7009a280 0x7f6c70146e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:35.960 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.949+0000 7f6c6effd700 1 --2- 172.21.15.40:0/3387154340 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6c7009bc40 0x7f6c700b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:35.961 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.949+0000 7f6c6ffff700 1 -- 172.21.15.40:0/3387154340 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6c7009bc40 msgr2=0x7f6c700b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:35.961 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.949+0000 7f6c6ffff700 1 --2- 172.21.15.40:0/3387154340 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6c7009bc40 0x7f6c700b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.961 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.949+0000 7f6c6ffff700 1 -- 172.21.15.40:0/3387154340 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6c7009a280 msgr2=0x7f6c70146e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:35.961 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.949+0000 7f6c6ffff700 1 --2- 172.21.15.40:0/3387154340 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6c7009a280 0x7f6c70146e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.961 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.949+0000 7f6c6ffff700 1 -- 172.21.15.40:0/3387154340 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c70064c90 con 0x7f6c700b88c0 2020-11-20T13:56:35.962 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.949+0000 7f6c6ffff700 1 --2- 172.21.15.40:0/3387154340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c700b88c0 0x7f6c70141e80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f6c680032b0 tx=0x7f6c68016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:35.962 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.950+0000 7f6c6cff9700 1 -- 172.21.15.40:0/3387154340 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c680105b0 con 0x7f6c700b88c0 2020-11-20T13:56:35.962 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.950+0000 7f6c6cff9700 1 -- 172.21.15.40:0/3387154340 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c68010710 con 0x7f6c700b88c0 2020-11-20T13:56:35.962 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.950+0000 7f6c75d58700 1 -- 172.21.15.40:0/3387154340 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c70149410 con 0x7f6c700b88c0 2020-11-20T13:56:35.962 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.950+0000 7f6c6cff9700 1 -- 172.21.15.40:0/3387154340 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c68010a70 con 0x7f6c700b88c0 2020-11-20T13:56:35.963 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.950+0000 7f6c75d58700 1 -- 172.21.15.40:0/3387154340 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c70142530 con 0x7f6c700b88c0 2020-11-20T13:56:35.963 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.951+0000 7f6c75d58700 1 -- 172.21.15.40:0/3387154340 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c70149720 con 0x7f6c700b88c0 2020-11-20T13:56:35.963 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.951+0000 7f6c6cff9700 1 -- 172.21.15.40:0/3387154340 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f6c68026030 con 0x7f6c700b88c0 2020-11-20T13:56:35.963 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.951+0000 7f6c6cff9700 1 --2- 172.21.15.40:0/3387154340 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f6c50051320 0x7f6c500537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:35.963 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.951+0000 7f6c6cff9700 1 -- 172.21.15.40:0/3387154340 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f6c6806a210 con 0x7f6c700b88c0 2020-11-20T13:56:35.964 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.951+0000 7f6c6f7fe700 1 --2- 172.21.15.40:0/3387154340 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f6c50051320 0x7f6c500537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:35.964 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.952+0000 7f6c6f7fe700 1 --2- 172.21.15.40:0/3387154340 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f6c50051320 0x7f6c500537e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f6c6000d000 tx=0x7f6c60007ad0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:35.964 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:35.955+0000 7f6c6cff9700 1 -- 172.21.15.40:0/3387154340 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f6c70149720 con 0x7f6c700b88c0 2020-11-20T13:56:36.109 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.109+0000 7f6c75d58700 1 -- 172.21.15.40:0/3387154340 --> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6c70064e20 con 0x7f6c50051320 2020-11-20T13:56:36.111 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.112+0000 7f6c6cff9700 1 -- 172.21.15.40:0/3387154340 <== mgr.4100 v2:172.21.15.116:6800/22233 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+82409 (secure 0 0 0) 0x7f6c70064e20 con 0x7f6c50051320 2020-11-20T13:56:36.113 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:56:36.114 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.113+0000 7f6c75d58700 1 -- 172.21.15.40:0/3387154340 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f6c50051320 msgr2=0x7f6c500537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:36.114 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.113+0000 7f6c75d58700 1 --2- 172.21.15.40:0/3387154340 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f6c50051320 0x7f6c500537e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f6c6000d000 tx=0x7f6c60007ad0).stop 2020-11-20T13:56:36.114 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.113+0000 7f6c75d58700 1 -- 172.21.15.40:0/3387154340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c700b88c0 msgr2=0x7f6c70141e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:36.115 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.113+0000 7f6c75d58700 1 --2- 172.21.15.40:0/3387154340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c700b88c0 0x7f6c70141e80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f6c680032b0 tx=0x7f6c68016000).stop 2020-11-20T13:56:36.115 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.113+0000 7f6c75d58700 1 -- 172.21.15.40:0/3387154340 shutdown_connections 2020-11-20T13:56:36.115 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.113+0000 7f6c75d58700 1 --2- 172.21.15.40:0/3387154340 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f6c50051320 0x7f6c500537e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.115 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.113+0000 7f6c75d58700 1 --2- 172.21.15.40:0/3387154340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c700b88c0 0x7f6c70141e80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.116 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.113+0000 7f6c75d58700 1 --2- 172.21.15.40:0/3387154340 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6c7009a280 0x7f6c70146e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.116 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.113+0000 7f6c75d58700 1 --2- 172.21.15.40:0/3387154340 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6c7009bc40 0x7f6c700b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.116 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.113+0000 7f6c75d58700 1 -- 172.21.15.40:0/3387154340 >> 172.21.15.40:0/3387154340 conn(0x7f6c700ba380 msgr2=0x7f6c700b6330 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:36.116 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.113+0000 7f6c75d58700 1 -- 172.21.15.40:0/3387154340 shutdown_connections 2020-11-20T13:56:36.117 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.113+0000 7f6c75d58700 1 -- 172.21.15.40:0/3387154340 wait complete. 2020-11-20T13:56:36.117 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2020-11-20T13:56:36.124 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2020-11-20T13:56:34.296512+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2304,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8390720,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746584000,"statfs":{"total":773094113280,"available":764502016000,"internally_reserved":8589934592,"allocated":2097152,"data_stored":826728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.258080"},"pg_stats":[{"pgid":"4.2","version":"0'0","reported_seq":"11","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-20T13:56:25.053797+0000","last_change":"2020-11-20T13:56:25.052346+0000","last_active":"2020-11-20T13:56:25.053797+0000","last_peered":"2020-11-20T13:56:25.053797+0000","last_clean":"2020-11-20T13:56:25.053797+0000","last_became_active":"2020-11-20T13:56:25.051763+0000","last_became_peered":"2020-11-20T13:56:25.051763+0000","last_unstale":"2020-11-20T13:56:25.053797+0000","last_undegraded":"2020-11-20T13:56:25.053797+0000","last_fullsized":"2020-11-20T13:56:25.053797+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:24.028995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:24.028995+0000","last_clean_scrub_stamp":"2020-11-20T13:56:24.028995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"15","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-20T13:56:25.044177+0000","last_change":"2020-11-20T13:56:19.143670+0000","last_active":"2020-11-20T13:56:25.044177+0000","last_peered":"2020-11-20T13:56:25.044177+0000","last_clean":"2020-11-20T13:56:25.044177+0000","last_became_active":"2020-11-20T13:56:19.143154+0000","last_became_peered":"2020-11-20T13:56:19.143154+0000","last_unstale":"2020-11-20T13:56:25.044177+0000","last_undegraded":"2020-11-20T13:56:25.044177+0000","last_fullsized":"2020-11-20T13:56:25.044177+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:17.989825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:17.989825+0000","last_clean_scrub_stamp":"2020-11-20T13:56:17.989825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"24'6","reported_seq":"18","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:56:25.450277+0000","last_change":"2020-11-20T13:56:22.036494+0000","last_active":"2020-11-20T13:56:25.450277+0000","last_peered":"2020-11-20T13:56:25.450277+0000","last_clean":"2020-11-20T13:56:25.450277+0000","last_became_active":"2020-11-20T13:56:22.036111+0000","last_became_peered":"2020-11-20T13:56:22.036111+0000","last_unstale":"2020-11-20T13:56:25.450277+0000","last_undegraded":"2020-11-20T13:56:25.450277+0000","last_fullsized":"2020-11-20T13:56:25.450277+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:21.011224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:21.011224+0000","last_clean_scrub_stamp":"2020-11-20T13:56:21.011224+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":"11","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-20T13:56:25.096170+0000","last_change":"2020-11-20T13:56:25.095407+0000","last_active":"2020-11-20T13:56:25.096170+0000","last_peered":"2020-11-20T13:56:25.096170+0000","last_clean":"2020-11-20T13:56:25.096170+0000","last_became_active":"2020-11-20T13:56:25.094738+0000","last_became_peered":"2020-11-20T13:56:25.094738+0000","last_unstale":"2020-11-20T13:56:25.096170+0000","last_undegraded":"2020-11-20T13:56:25.096170+0000","last_fullsized":"2020-11-20T13:56:25.096170+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:24.028995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:24.028995+0000","last_clean_scrub_stamp":"2020-11-20T13:56:24.028995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"15","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-20T13:56:25.040550+0000","last_change":"2020-11-20T13:56:19.010697+0000","last_active":"2020-11-20T13:56:25.040550+0000","last_peered":"2020-11-20T13:56:25.040550+0000","last_clean":"2020-11-20T13:56:25.040550+0000","last_became_active":"2020-11-20T13:56:19.010404+0000","last_became_peered":"2020-11-20T13:56:19.010404+0000","last_unstale":"2020-11-20T13:56:25.040550+0000","last_undegraded":"2020-11-20T13:56:25.040550+0000","last_fullsized":"2020-11-20T13:56:25.040550+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:17.989825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:17.989825+0000","last_clean_scrub_stamp":"2020-11-20T13:56:17.989825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"24'5","reported_seq":"17","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:56:25.447431+0000","last_change":"2020-11-20T13:56:22.038645+0000","last_active":"2020-11-20T13:56:25.447431+0000","last_peered":"2020-11-20T13:56:25.447431+0000","last_clean":"2020-11-20T13:56:25.447431+0000","last_became_active":"2020-11-20T13:56:22.038261+0000","last_became_peered":"2020-11-20T13:56:22.038261+0000","last_unstale":"2020-11-20T13:56:25.447431+0000","last_undegraded":"2020-11-20T13:56:25.447431+0000","last_fullsized":"2020-11-20T13:56:25.447431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:21.011224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:21.011224+0000","last_clean_scrub_stamp":"2020-11-20T13:56:21.011224+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":"11","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-20T13:56:25.051161+0000","last_change":"2020-11-20T13:56:25.049944+0000","last_active":"2020-11-20T13:56:25.051161+0000","last_peered":"2020-11-20T13:56:25.051161+0000","last_clean":"2020-11-20T13:56:25.051161+0000","last_became_active":"2020-11-20T13:56:25.049257+0000","last_became_peered":"2020-11-20T13:56:25.049257+0000","last_unstale":"2020-11-20T13:56:25.051161+0000","last_undegraded":"2020-11-20T13:56:25.051161+0000","last_fullsized":"2020-11-20T13:56:25.051161+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:24.028995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:24.028995+0000","last_clean_scrub_stamp":"2020-11-20T13:56:24.028995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"15","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-20T13:56:25.093139+0000","last_change":"2020-11-20T13:56:19.010270+0000","last_active":"2020-11-20T13:56:25.093139+0000","last_peered":"2020-11-20T13:56:25.093139+0000","last_clean":"2020-11-20T13:56:25.093139+0000","last_became_active":"2020-11-20T13:56:19.009804+0000","last_became_peered":"2020-11-20T13:56:19.009804+0000","last_unstale":"2020-11-20T13:56:25.093139+0000","last_undegraded":"2020-11-20T13:56:25.093139+0000","last_fullsized":"2020-11-20T13:56:25.093139+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:17.989825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:17.989825+0000","last_clean_scrub_stamp":"2020-11-20T13:56:17.989825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"24'8","reported_seq":"20","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:56:25.444883+0000","last_change":"2020-11-20T13:56:22.039123+0000","last_active":"2020-11-20T13:56:25.444883+0000","last_peered":"2020-11-20T13:56:25.444883+0000","last_clean":"2020-11-20T13:56:25.444883+0000","last_became_active":"2020-11-20T13:56:22.038692+0000","last_became_peered":"2020-11-20T13:56:22.038692+0000","last_unstale":"2020-11-20T13:56:25.444883+0000","last_undegraded":"2020-11-20T13:56:25.444883+0000","last_fullsized":"2020-11-20T13:56:25.444883+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:21.011224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:21.011224+0000","last_clean_scrub_stamp":"2020-11-20T13:56:21.011224+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":"11","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-20T13:56:25.056607+0000","last_change":"2020-11-20T13:56:25.050749+0000","last_active":"2020-11-20T13:56:25.056607+0000","last_peered":"2020-11-20T13:56:25.056607+0000","last_clean":"2020-11-20T13:56:25.056607+0000","last_became_active":"2020-11-20T13:56:25.049653+0000","last_became_peered":"2020-11-20T13:56:25.049653+0000","last_unstale":"2020-11-20T13:56:25.056607+0000","last_undegraded":"2020-11-20T13:56:25.056607+0000","last_fullsized":"2020-11-20T13:56:25.056607+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:24.028995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:24.028995+0000","last_clean_scrub_stamp":"2020-11-20T13:56:24.028995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"15","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-20T13:56:25.040571+0000","last_change":"2020-11-20T13:56:19.012731+0000","last_active":"2020-11-20T13:56:25.040571+0000","last_peered":"2020-11-20T13:56:25.040571+0000","last_clean":"2020-11-20T13:56:25.040571+0000","last_became_active":"2020-11-20T13:56:19.012477+0000","last_became_peered":"2020-11-20T13:56:19.012477+0000","last_unstale":"2020-11-20T13:56:25.040571+0000","last_undegraded":"2020-11-20T13:56:25.040571+0000","last_fullsized":"2020-11-20T13:56:25.040571+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:17.989825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:17.989825+0000","last_clean_scrub_stamp":"2020-11-20T13:56:17.989825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"20","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-20T13:56:25.040746+0000","last_change":"2020-11-20T13:56:15.047948+0000","last_active":"2020-11-20T13:56:25.040746+0000","last_peered":"2020-11-20T13:56:25.040746+0000","last_clean":"2020-11-20T13:56:25.040746+0000","last_became_active":"2020-11-20T13:56:15.047659+0000","last_became_peered":"2020-11-20T13:56:15.047659+0000","last_unstale":"2020-11-20T13:56:25.040746+0000","last_undegraded":"2020-11-20T13:56:25.040746+0000","last_fullsized":"2020-11-20T13:56:25.040746+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":6,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:08.473667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:08.473667+0000","last_clean_scrub_stamp":"2020-11-20T13:56:08.473667+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"24'1","reported_seq":"13","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:56:25.443619+0000","last_change":"2020-11-20T13:56:22.036039+0000","last_active":"2020-11-20T13:56:25.443619+0000","last_peered":"2020-11-20T13:56:25.443619+0000","last_clean":"2020-11-20T13:56:25.443619+0000","last_became_active":"2020-11-20T13:56:22.035637+0000","last_became_peered":"2020-11-20T13:56:22.035637+0000","last_unstale":"2020-11-20T13:56:25.443619+0000","last_undegraded":"2020-11-20T13:56:25.443619+0000","last_fullsized":"2020-11-20T13:56:25.443619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:21.011224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:21.011224+0000","last_clean_scrub_stamp":"2020-11-20T13:56:21.011224+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":"11","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-20T13:56:25.097051+0000","last_change":"2020-11-20T13:56:25.095551+0000","last_active":"2020-11-20T13:56:25.097051+0000","last_peered":"2020-11-20T13:56:25.097051+0000","last_clean":"2020-11-20T13:56:25.097051+0000","last_became_active":"2020-11-20T13:56:25.094843+0000","last_became_peered":"2020-11-20T13:56:25.094843+0000","last_unstale":"2020-11-20T13:56:25.097051+0000","last_undegraded":"2020-11-20T13:56:25.097051+0000","last_fullsized":"2020-11-20T13:56:25.097051+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:24.028995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:24.028995+0000","last_clean_scrub_stamp":"2020-11-20T13:56:24.028995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"15","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-20T13:56:25.040820+0000","last_change":"2020-11-20T13:56:19.008877+0000","last_active":"2020-11-20T13:56:25.040820+0000","last_peered":"2020-11-20T13:56:25.040820+0000","last_clean":"2020-11-20T13:56:25.040820+0000","last_became_active":"2020-11-20T13:56:19.008558+0000","last_became_peered":"2020-11-20T13:56:19.008558+0000","last_unstale":"2020-11-20T13:56:25.040820+0000","last_undegraded":"2020-11-20T13:56:25.040820+0000","last_fullsized":"2020-11-20T13:56:25.040820+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:17.989825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:17.989825+0000","last_clean_scrub_stamp":"2020-11-20T13:56:17.989825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"12","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-20T13:56:25.041657+0000","last_change":"2020-11-20T13:56:22.031448+0000","last_active":"2020-11-20T13:56:25.041657+0000","last_peered":"2020-11-20T13:56:25.041657+0000","last_clean":"2020-11-20T13:56:25.041657+0000","last_became_active":"2020-11-20T13:56:22.031157+0000","last_became_peered":"2020-11-20T13:56:22.031157+0000","last_unstale":"2020-11-20T13:56:25.041657+0000","last_undegraded":"2020-11-20T13:56:25.041657+0000","last_fullsized":"2020-11-20T13:56:25.041657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:21.011224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:21.011224+0000","last_clean_scrub_stamp":"2020-11-20T13:56:21.011224+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"11","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-20T13:56:25.050669+0000","last_change":"2020-11-20T13:56:25.049467+0000","last_active":"2020-11-20T13:56:25.050669+0000","last_peered":"2020-11-20T13:56:25.050669+0000","last_clean":"2020-11-20T13:56:25.050669+0000","last_became_active":"2020-11-20T13:56:25.048869+0000","last_became_peered":"2020-11-20T13:56:25.048869+0000","last_unstale":"2020-11-20T13:56:25.050669+0000","last_undegraded":"2020-11-20T13:56:25.050669+0000","last_fullsized":"2020-11-20T13:56:25.050669+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:24.028995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:24.028995+0000","last_clean_scrub_stamp":"2020-11-20T13:56:24.028995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"15","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-20T13:56:25.040938+0000","last_change":"2020-11-20T13:56:19.144378+0000","last_active":"2020-11-20T13:56:25.040938+0000","last_peered":"2020-11-20T13:56:25.040938+0000","last_clean":"2020-11-20T13:56:25.040938+0000","last_became_active":"2020-11-20T13:56:19.144086+0000","last_became_peered":"2020-11-20T13:56:19.144086+0000","last_unstale":"2020-11-20T13:56:25.040938+0000","last_undegraded":"2020-11-20T13:56:25.040938+0000","last_fullsized":"2020-11-20T13:56:25.040938+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:17.989825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:17.989825+0000","last_clean_scrub_stamp":"2020-11-20T13:56:17.989825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"24'8","reported_seq":"20","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:56:25.455025+0000","last_change":"2020-11-20T13:56:22.039342+0000","last_active":"2020-11-20T13:56:25.455025+0000","last_peered":"2020-11-20T13:56:25.455025+0000","last_clean":"2020-11-20T13:56:25.455025+0000","last_became_active":"2020-11-20T13:56:22.038953+0000","last_became_peered":"2020-11-20T13:56:22.038953+0000","last_unstale":"2020-11-20T13:56:25.455025+0000","last_undegraded":"2020-11-20T13:56:25.455025+0000","last_fullsized":"2020-11-20T13:56:25.455025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:21.011224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:21.011224+0000","last_clean_scrub_stamp":"2020-11-20T13:56:21.011224+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1535,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"11","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-20T13:56:25.051029+0000","last_change":"2020-11-20T13:56:25.050098+0000","last_active":"2020-11-20T13:56:25.051029+0000","last_peered":"2020-11-20T13:56:25.051029+0000","last_clean":"2020-11-20T13:56:25.051029+0000","last_became_active":"2020-11-20T13:56:25.049451+0000","last_became_peered":"2020-11-20T13:56:25.049451+0000","last_unstale":"2020-11-20T13:56:25.051029+0000","last_undegraded":"2020-11-20T13:56:25.051029+0000","last_fullsized":"2020-11-20T13:56:25.051029+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:24.028995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:24.028995+0000","last_clean_scrub_stamp":"2020-11-20T13:56:24.028995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"24'7","reported_seq":"19","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:56:25.450024+0000","last_change":"2020-11-20T13:56:22.033990+0000","last_active":"2020-11-20T13:56:25.450024+0000","last_peered":"2020-11-20T13:56:25.450024+0000","last_clean":"2020-11-20T13:56:25.450024+0000","last_became_active":"2020-11-20T13:56:22.033658+0000","last_became_peered":"2020-11-20T13:56:22.033658+0000","last_unstale":"2020-11-20T13:56:25.450024+0000","last_undegraded":"2020-11-20T13:56:25.450024+0000","last_fullsized":"2020-11-20T13:56:25.450024+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:21.011224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:21.011224+0000","last_clean_scrub_stamp":"2020-11-20T13:56:21.011224+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":575,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"15","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-20T13:56:25.044942+0000","last_change":"2020-11-20T13:56:19.013591+0000","last_active":"2020-11-20T13:56:25.044942+0000","last_peered":"2020-11-20T13:56:25.044942+0000","last_clean":"2020-11-20T13:56:25.044942+0000","last_became_active":"2020-11-20T13:56:19.013128+0000","last_became_peered":"2020-11-20T13:56:19.013128+0000","last_unstale":"2020-11-20T13:56:25.044942+0000","last_undegraded":"2020-11-20T13:56:25.044942+0000","last_fullsized":"2020-11-20T13:56:25.044942+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:17.989825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:17.989825+0000","last_clean_scrub_stamp":"2020-11-20T13:56:17.989825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"11","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-20T13:56:25.049208+0000","last_change":"2020-11-20T13:56:25.047910+0000","last_active":"2020-11-20T13:56:25.049208+0000","last_peered":"2020-11-20T13:56:25.049208+0000","last_clean":"2020-11-20T13:56:25.049208+0000","last_became_active":"2020-11-20T13:56:25.047294+0000","last_became_peered":"2020-11-20T13:56:25.047294+0000","last_unstale":"2020-11-20T13:56:25.049208+0000","last_undegraded":"2020-11-20T13:56:25.049208+0000","last_fullsized":"2020-11-20T13:56:25.049208+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:24.028995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:24.028995+0000","last_clean_scrub_stamp":"2020-11-20T13:56:24.028995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"14","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-20T13:56:25.044582+0000","last_change":"2020-11-20T13:56:19.015762+0000","last_active":"2020-11-20T13:56:25.044582+0000","last_peered":"2020-11-20T13:56:25.044582+0000","last_clean":"2020-11-20T13:56:25.044582+0000","last_became_active":"2020-11-20T13:56:19.015422+0000","last_became_peered":"2020-11-20T13:56:19.015422+0000","last_unstale":"2020-11-20T13:56:25.044582+0000","last_undegraded":"2020-11-20T13:56:25.044582+0000","last_fullsized":"2020-11-20T13:56:25.044582+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:17.989825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:17.989825+0000","last_clean_scrub_stamp":"2020-11-20T13:56:17.989825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"24'4","reported_seq":"16","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:56:25.445741+0000","last_change":"2020-11-20T13:56:22.039130+0000","last_active":"2020-11-20T13:56:25.445741+0000","last_peered":"2020-11-20T13:56:25.445741+0000","last_clean":"2020-11-20T13:56:25.445741+0000","last_became_active":"2020-11-20T13:56:22.038719+0000","last_became_peered":"2020-11-20T13:56:22.038719+0000","last_unstale":"2020-11-20T13:56:25.445741+0000","last_undegraded":"2020-11-20T13:56:25.445741+0000","last_fullsized":"2020-11-20T13:56:25.445741+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:56:21.011224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:56:21.011224+0000","last_clean_scrub_stamp":"2020-11-20T13:56:21.011224+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":32,"acting":32,"num_store_stats":8},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048840,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323000,"statfs":{"total":96636764160,"available":95562752000,"internally_reserved":1073741824,"allocated":262144,"data_stored":103340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1419999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3999999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048832,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323008,"statfs":{"total":96636764160,"available":95562760192,"internally_reserved":1073741824,"allocated":253952,"data_stored":102765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0860000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.319}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3779999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048844,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322996,"statfs":{"total":96636764160,"available":95562747904,"internally_reserved":1073741824,"allocated":266240,"data_stored":104300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.103}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2769999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2529999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048836,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323004,"statfs":{"total":96636764160,"available":95562756096,"internally_reserved":1073741824,"allocated":258048,"data_stored":102799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2150000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.149}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1739999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048844,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322996,"statfs":{"total":96636764160,"available":95562747904,"internally_reserved":1073741824,"allocated":266240,"data_stored":104300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2130000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048848,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322992,"statfs":{"total":96636764160,"available":95562743808,"internally_reserved":1073741824,"allocated":270336,"data_stored":103500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.452}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1160000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2290000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048836,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323004,"statfs":{"total":96636764160,"available":95562756096,"internally_reserved":1073741824,"allocated":258048,"data_stored":102799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.198}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.087}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.492}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048840,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323000,"statfs":{"total":96636764160,"available":95562752000,"internally_reserved":1073741824,"allocated":262144,"data_stored":102925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1479999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3160000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.246}]}]}]}} 2020-11-20T13:56:36.126 INFO:tasks.ceph.ceph_manager.ceph:clean! 2020-11-20T13:56:36.127 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2020-11-20T13:56:36.127 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2020-11-20T13:56:36.129 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early health --format=json 2020-11-20T13:56:36.242 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.242+0000 7f54d047e700 1 Processor -- start 2020-11-20T13:56:36.243 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.242+0000 7f54d047e700 1 -- start start 2020-11-20T13:56:36.245 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.243+0000 7f54d047e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f54c80b6c60 0x7f54c80b2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:36.246 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.243+0000 7f54d047e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f54c8143b40 0x7f54c8143fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:36.246 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.243+0000 7f54d047e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54c80b2e50 con 0x7f54c80b22e0 2020-11-20T13:56:36.246 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.243+0000 7f54d047e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54c80b30e0 con 0x7f54c8143b40 2020-11-20T13:56:36.246 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.243+0000 7f54d047e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54c80b5820 con 0x7f54c80b6c60 2020-11-20T13:56:36.247 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.243+0000 7f54cea1b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f54c8143b40 0x7f54c8143fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:36.247 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.243+0000 7f54cea1b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f54c8143b40 0x7f54c8143fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54968/0 (socket says 172.21.15.40:54968) 2020-11-20T13:56:36.247 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.243+0000 7f54cea1b700 1 -- 172.21.15.40:0/208726147 learned_addr learned my addr 172.21.15.40:0/208726147 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:36.247 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.244+0000 7f54ce21a700 1 --2- 172.21.15.40:0/208726147 >> v2:172.21.15.116:3300/0 conn(0x7f54c80b6c60 0x7f54c80b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:36.247 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.244+0000 7f54cea1b700 1 -- 172.21.15.40:0/208726147 >> v2:172.21.15.116:3300/0 conn(0x7f54c80b6c60 msgr2=0x7f54c80b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:36.248 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.244+0000 7f54cea1b700 1 --2- 172.21.15.40:0/208726147 >> v2:172.21.15.116:3300/0 conn(0x7f54c80b6c60 0x7f54c80b2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.248 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.244+0000 7f54cea1b700 1 -- 172.21.15.40:0/208726147 >> v1:172.21.15.116:6789/0 conn(0x7f54c80b22e0 legacy=0x7f54c80b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:36.249 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.244+0000 7f54cea1b700 1 -- 172.21.15.40:0/208726147 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c80b5ac0 con 0x7f54c8143b40 2020-11-20T13:56:36.249 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.244+0000 7f54ce21a700 1 --2- 172.21.15.40:0/208726147 >> v2:172.21.15.116:3300/0 conn(0x7f54c80b6c60 0x7f54c80b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:36.249 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.244+0000 7f54cea1b700 1 --2- 172.21.15.40:0/208726147 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54c8143b40 0x7f54c8143fd0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f54c8061350 tx=0x7f54c0009270).ready entity=mon.0 client_cookie=c86829273b2e3854 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:36.249 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.244+0000 7f54cd218700 1 -- 172.21.15.40:0/208726147 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54c000bbc0 con 0x7f54c8143b40 2020-11-20T13:56:36.249 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.244+0000 7f54cd218700 1 -- 172.21.15.40:0/208726147 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54c000bd20 con 0x7f54c8143b40 2020-11-20T13:56:36.250 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.244+0000 7f54cd218700 1 -- 172.21.15.40:0/208726147 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54c0011400 con 0x7f54c8143b40 2020-11-20T13:56:36.250 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.244+0000 7f54d047e700 1 -- 172.21.15.40:0/208726147 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54c8143b40 msgr2=0x7f54c8143fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:36.250 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.244+0000 7f54d047e700 1 --2- 172.21.15.40:0/208726147 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54c8143b40 0x7f54c8143fd0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f54c8061350 tx=0x7f54c0009270).stop 2020-11-20T13:56:36.250 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.244+0000 7f54d047e700 1 -- 172.21.15.40:0/208726147 shutdown_connections 2020-11-20T13:56:36.250 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.244+0000 7f54d047e700 1 --2- 172.21.15.40:0/208726147 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54c8143b40 0x7f54c8143fd0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.251 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.244+0000 7f54d047e700 1 --2- 172.21.15.40:0/208726147 >> v2:172.21.15.116:3300/0 conn(0x7f54c80b6c60 0x7f54c80b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.251 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.244+0000 7f54d047e700 1 -- 172.21.15.40:0/208726147 >> 172.21.15.40:0/208726147 conn(0x7f54c80ba200 msgr2=0x7f54c80b61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:36.251 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.245+0000 7f54d047e700 1 -- 172.21.15.40:0/208726147 shutdown_connections 2020-11-20T13:56:36.251 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.245+0000 7f54d047e700 1 -- 172.21.15.40:0/208726147 wait complete. 2020-11-20T13:56:36.251 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.246+0000 7f54d047e700 1 Processor -- start 2020-11-20T13:56:36.252 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.246+0000 7f54d047e700 1 -- start start 2020-11-20T13:56:36.254 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.246+0000 7f54d047e700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f54c80b22e0 0x7f54c80b4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:36.254 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.246+0000 7f54d047e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54c80b6c60 0x7f54c81562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:36.254 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.246+0000 7f54d047e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54c8143b40 0x7f54c815a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:36.254 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.246+0000 7f54d047e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c8156cf0 con 0x7f54c80b6c60 2020-11-20T13:56:36.255 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.246+0000 7f54d047e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c8156f90 con 0x7f54c8143b40 2020-11-20T13:56:36.255 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.246+0000 7f54d047e700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54c8157230 con 0x7f54c80b22e0 2020-11-20T13:56:36.255 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.246+0000 7f54cda19700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54c80b6c60 0x7f54c81562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:36.255 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.246+0000 7f54cda19700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54c80b6c60 0x7f54c81562f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54976/0 (socket says 172.21.15.40:54976) 2020-11-20T13:56:36.255 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.246+0000 7f54cda19700 1 -- 172.21.15.40:0/44734105 learned_addr learned my addr 172.21.15.40:0/44734105 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:36.256 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.246+0000 7f54cea1b700 1 --2- 172.21.15.40:0/44734105 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54c8143b40 0x7f54c815a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:36.256 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.246+0000 7f54ce21a700 1 --2- 172.21.15.40:0/44734105 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f54c80b22e0 0x7f54c80b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:36.256 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.247+0000 7f54cda19700 1 -- 172.21.15.40:0/44734105 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f54c80b22e0 msgr2=0x7f54c80b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:36.256 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.247+0000 7f54cda19700 1 --2- 172.21.15.40:0/44734105 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f54c80b22e0 0x7f54c80b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.256 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.247+0000 7f54cda19700 1 -- 172.21.15.40:0/44734105 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54c8143b40 msgr2=0x7f54c815a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:36.257 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.247+0000 7f54cda19700 1 --2- 172.21.15.40:0/44734105 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54c8143b40 0x7f54c815a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.257 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.247+0000 7f54cda19700 1 -- 172.21.15.40:0/44734105 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c8146020 con 0x7f54c80b6c60 2020-11-20T13:56:36.257 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.247+0000 7f54cda19700 1 --2- 172.21.15.40:0/44734105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54c80b6c60 0x7f54c81562f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f54bc001f50 tx=0x7f54bc007bd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:36.257 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.247+0000 7f54bb7fe700 1 -- 172.21.15.40:0/44734105 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54bc013410 con 0x7f54c80b6c60 2020-11-20T13:56:36.257 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.247+0000 7f54bb7fe700 1 -- 172.21.15.40:0/44734105 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54bc013570 con 0x7f54c80b6c60 2020-11-20T13:56:36.258 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.247+0000 7f54bb7fe700 1 -- 172.21.15.40:0/44734105 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54bc0138d0 con 0x7f54c80b6c60 2020-11-20T13:56:36.258 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.247+0000 7f54d047e700 1 -- 172.21.15.40:0/44734105 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54c8159ad0 con 0x7f54c80b6c60 2020-11-20T13:56:36.258 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.247+0000 7f54d047e700 1 -- 172.21.15.40:0/44734105 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54c81569a0 con 0x7f54c80b6c60 2020-11-20T13:56:36.259 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.248+0000 7f54bb7fe700 1 -- 172.21.15.40:0/44734105 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f54bc010030 con 0x7f54c80b6c60 2020-11-20T13:56:36.259 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.248+0000 7f54d047e700 1 -- 172.21.15.40:0/44734105 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54c815a930 con 0x7f54c80b6c60 2020-11-20T13:56:36.259 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.248+0000 7f54bb7fe700 1 --2- 172.21.15.40:0/44734105 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f54a8051320 0x7f54a80537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:36.260 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.248+0000 7f54bb7fe700 1 -- 172.21.15.40:0/44734105 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f54bc068db0 con 0x7f54c80b6c60 2020-11-20T13:56:36.260 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.249+0000 7f54ce21a700 1 --2- 172.21.15.40:0/44734105 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f54a8051320 0x7f54a80537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:36.261 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.249+0000 7f54ce21a700 1 --2- 172.21.15.40:0/44734105 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f54a8051320 0x7f54a80537e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f54c400d000 tx=0x7f54c4008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:36.261 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.252+0000 7f54bb7fe700 1 -- 172.21.15.40:0/44734105 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f54bc041080 con 0x7f54c80b6c60 2020-11-20T13:56:36.475 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.475+0000 7f54d047e700 1 -- 172.21.15.40:0/44734105 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f54c80686e0 con 0x7f54c80b6c60 2020-11-20T13:56:36.476 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.475+0000 7f54bb7fe700 1 -- 172.21.15.40:0/44734105 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+45 (secure 0 0 0) 0x7f54bc0446a0 con 0x7f54c80b6c60 2020-11-20T13:56:36.476 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:56:36.478 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.477+0000 7f54d047e700 1 -- 172.21.15.40:0/44734105 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f54a8051320 msgr2=0x7f54a80537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:36.478 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.477+0000 7f54d047e700 1 --2- 172.21.15.40:0/44734105 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f54a8051320 0x7f54a80537e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f54c400d000 tx=0x7f54c4008040).stop 2020-11-20T13:56:36.478 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.477+0000 7f54d047e700 1 -- 172.21.15.40:0/44734105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54c80b6c60 msgr2=0x7f54c81562f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:36.479 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.477+0000 7f54d047e700 1 --2- 172.21.15.40:0/44734105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54c80b6c60 0x7f54c81562f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f54bc001f50 tx=0x7f54bc007bd0).stop 2020-11-20T13:56:36.479 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.477+0000 7f54d047e700 1 -- 172.21.15.40:0/44734105 shutdown_connections 2020-11-20T13:56:36.479 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.477+0000 7f54d047e700 1 --2- 172.21.15.40:0/44734105 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f54a8051320 0x7f54a80537e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.479 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.477+0000 7f54d047e700 1 --2- 172.21.15.40:0/44734105 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f54c80b22e0 0x7f54c80b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.479 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.477+0000 7f54d047e700 1 --2- 172.21.15.40:0/44734105 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54c8143b40 0x7f54c815a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.480 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.477+0000 7f54d047e700 1 --2- 172.21.15.40:0/44734105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54c80b6c60 0x7f54c81562f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.480 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.477+0000 7f54d047e700 1 -- 172.21.15.40:0/44734105 >> 172.21.15.40:0/44734105 conn(0x7f54c80ba200 msgr2=0x7f54c80ae730 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:36.480 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.477+0000 7f54d047e700 1 -- 172.21.15.40:0/44734105 shutdown_connections 2020-11-20T13:56:36.480 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.477+0000 7f54d047e700 1 -- 172.21.15.40:0/44734105 wait complete. 2020-11-20T13:56:36.487 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2020-11-20T13:56:36.488 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2020-11-20T13:56:36.488 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-20T13:56:36.602 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.602+0000 7f26f1aa1700 1 Processor -- start 2020-11-20T13:56:36.603 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.602+0000 7f26f1aa1700 1 -- start start 2020-11-20T13:56:36.605 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.603+0000 7f26f1aa1700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f26ec09a280 0x7f26ec0b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:36.606 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.603+0000 7f26f1aa1700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f26ec09bc40 0x7f26ec09c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:36.606 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.603+0000 7f26f1aa1700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26ec0b6be0 con 0x7f26ec0b88c0 2020-11-20T13:56:36.606 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.603+0000 7f26f1aa1700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26ec099000 con 0x7f26ec09bc40 2020-11-20T13:56:36.606 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.603+0000 7f26f1aa1700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26ec09c570 con 0x7f26ec09a280 2020-11-20T13:56:36.607 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.603+0000 7f26ebfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f26ec09bc40 0x7f26ec09c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:36.608 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.603+0000 7f26ebfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f26ec09bc40 0x7f26ec09c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54984/0 (socket says 172.21.15.40:54984) 2020-11-20T13:56:36.608 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.603+0000 7f26ebfff700 1 -- 172.21.15.40:0/3603988085 learned_addr learned my addr 172.21.15.40:0/3603988085 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:36.608 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.603+0000 7f26eaffd700 1 --2- 172.21.15.40:0/3603988085 >> v2:172.21.15.116:3300/0 conn(0x7f26ec09a280 0x7f26ec0b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:36.608 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.603+0000 7f26ebfff700 1 -- 172.21.15.40:0/3603988085 >> v2:172.21.15.116:3300/0 conn(0x7f26ec09a280 msgr2=0x7f26ec0b8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:36.608 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.603+0000 7f26ebfff700 1 --2- 172.21.15.40:0/3603988085 >> v2:172.21.15.116:3300/0 conn(0x7f26ec09a280 0x7f26ec0b8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.609 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.603+0000 7f26ebfff700 1 -- 172.21.15.40:0/3603988085 >> v1:172.21.15.116:6789/0 conn(0x7f26ec0b88c0 legacy=0x7f26ec09b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:36.609 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.603+0000 7f26ebfff700 1 -- 172.21.15.40:0/3603988085 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26ec0b4250 con 0x7f26ec09bc40 2020-11-20T13:56:36.609 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.604+0000 7f26ebfff700 1 --2- 172.21.15.40:0/3603988085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26ec09bc40 0x7f26ec09c080 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f26e40032b0 tx=0x7f26e4009270).ready entity=mon.0 client_cookie=6d5fa7ea10950013 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:36.609 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.604+0000 7f26ea7fc700 1 -- 172.21.15.40:0/3603988085 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f26e400bc00 con 0x7f26ec09bc40 2020-11-20T13:56:36.609 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.604+0000 7f26ea7fc700 1 -- 172.21.15.40:0/3603988085 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e400bd60 con 0x7f26ec09bc40 2020-11-20T13:56:36.611 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.604+0000 7f26ea7fc700 1 -- 172.21.15.40:0/3603988085 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f26e4011400 con 0x7f26ec09bc40 2020-11-20T13:56:36.611 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.604+0000 7f26f1aa1700 1 -- 172.21.15.40:0/3603988085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26ec09bc40 msgr2=0x7f26ec09c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:36.611 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.604+0000 7f26f1aa1700 1 --2- 172.21.15.40:0/3603988085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26ec09bc40 0x7f26ec09c080 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f26e40032b0 tx=0x7f26e4009270).stop 2020-11-20T13:56:36.611 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.604+0000 7f26f1aa1700 1 -- 172.21.15.40:0/3603988085 shutdown_connections 2020-11-20T13:56:36.612 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.604+0000 7f26f1aa1700 1 --2- 172.21.15.40:0/3603988085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26ec09bc40 0x7f26ec09c080 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.612 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.604+0000 7f26f1aa1700 1 --2- 172.21.15.40:0/3603988085 >> v2:172.21.15.116:3300/0 conn(0x7f26ec09a280 0x7f26ec0b8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.612 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.604+0000 7f26f1aa1700 1 -- 172.21.15.40:0/3603988085 >> 172.21.15.40:0/3603988085 conn(0x7f26ec0ba380 msgr2=0x7f26ec0b6350 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:36.613 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.604+0000 7f26f1aa1700 1 -- 172.21.15.40:0/3603988085 shutdown_connections 2020-11-20T13:56:36.613 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.604+0000 7f26f1aa1700 1 -- 172.21.15.40:0/3603988085 wait complete. 2020-11-20T13:56:36.613 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.605+0000 7f26f1aa1700 1 Processor -- start 2020-11-20T13:56:36.613 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.605+0000 7f26f1aa1700 1 -- start start 2020-11-20T13:56:36.616 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.606+0000 7f26f1aa1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26ec09a280 0x7f26ec060b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:36.616 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.606+0000 7f26f1aa1700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f26ec09bc40 0x7f26ec154720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:36.616 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.606+0000 7f26f1aa1700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26ec0b88c0 0x7f26ec13cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:36.618 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.606+0000 7f26f1aa1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ec0b25c0 con 0x7f26ec09a280 2020-11-20T13:56:36.618 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.606+0000 7f26f1aa1700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ec0b2860 con 0x7f26ec0b88c0 2020-11-20T13:56:36.618 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.606+0000 7f26f1aa1700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26ec13d470 con 0x7f26ec09bc40 2020-11-20T13:56:36.618 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.606+0000 7f26eb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26ec09a280 0x7f26ec060b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:36.618 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.606+0000 7f26eb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26ec09a280 0x7f26ec060b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54988/0 (socket says 172.21.15.40:54988) 2020-11-20T13:56:36.619 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.606+0000 7f26eb7fe700 1 -- 172.21.15.40:0/1807810262 learned_addr learned my addr 172.21.15.40:0/1807810262 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:36.619 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.606+0000 7f26eaffd700 1 --2- 172.21.15.40:0/1807810262 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f26ec09bc40 0x7f26ec154720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:36.619 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.606+0000 7f26ebfff700 1 --2- 172.21.15.40:0/1807810262 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26ec0b88c0 0x7f26ec13cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:36.619 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.606+0000 7f26eb7fe700 1 -- 172.21.15.40:0/1807810262 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f26ec09bc40 msgr2=0x7f26ec154720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:36.620 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.607+0000 7f26eb7fe700 1 --2- 172.21.15.40:0/1807810262 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f26ec09bc40 0x7f26ec154720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.620 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.607+0000 7f26eb7fe700 1 -- 172.21.15.40:0/1807810262 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26ec0b88c0 msgr2=0x7f26ec13cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:36.620 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.607+0000 7f26eb7fe700 1 --2- 172.21.15.40:0/1807810262 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26ec0b88c0 0x7f26ec13cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.620 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.607+0000 7f26eb7fe700 1 -- 172.21.15.40:0/1807810262 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26ec13ae80 con 0x7f26ec09a280 2020-11-20T13:56:36.620 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.607+0000 7f26eb7fe700 1 --2- 172.21.15.40:0/1807810262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26ec09a280 0x7f26ec060b40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f26ec0b73b0 tx=0x7f26dc007b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:36.621 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.607+0000 7f26e8ff9700 1 -- 172.21.15.40:0/1807810262 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f26dc007030 con 0x7f26ec09a280 2020-11-20T13:56:36.621 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.607+0000 7f26e8ff9700 1 -- 172.21.15.40:0/1807810262 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26dc007190 con 0x7f26ec09a280 2020-11-20T13:56:36.621 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.607+0000 7f26f1aa1700 1 -- 172.21.15.40:0/1807810262 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26ec13b0d0 con 0x7f26ec09a280 2020-11-20T13:56:36.621 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.607+0000 7f26e8ff9700 1 -- 172.21.15.40:0/1807810262 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f26dc0064e0 con 0x7f26ec09a280 2020-11-20T13:56:36.621 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.607+0000 7f26f1aa1700 1 -- 172.21.15.40:0/1807810262 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26ec14daf0 con 0x7f26ec09a280 2020-11-20T13:56:36.622 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.608+0000 7f26f1aa1700 1 -- 172.21.15.40:0/1807810262 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26ec14e180 con 0x7f26ec09a280 2020-11-20T13:56:36.622 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.608+0000 7f26e8ff9700 1 -- 172.21.15.40:0/1807810262 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f26dc010070 con 0x7f26ec09a280 2020-11-20T13:56:36.622 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.608+0000 7f26e8ff9700 1 --2- 172.21.15.40:0/1807810262 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f26cc051320 0x7f26cc0537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:36.623 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.608+0000 7f26e8ff9700 1 -- 172.21.15.40:0/1807810262 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f26dc0685b0 con 0x7f26ec09a280 2020-11-20T13:56:36.623 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.608+0000 7f26eaffd700 1 --2- 172.21.15.40:0/1807810262 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f26cc051320 0x7f26cc0537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:36.624 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.609+0000 7f26eaffd700 1 --2- 172.21.15.40:0/1807810262 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f26cc051320 0x7f26cc0537e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f26e000c000 tx=0x7f26e0008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:36.624 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.612+0000 7f26e8ff9700 1 -- 172.21.15.40:0/1807810262 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f26dc0407d0 con 0x7f26ec09a280 2020-11-20T13:56:36.774 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.774+0000 7f26f1aa1700 1 -- 172.21.15.40:0/1807810262 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26ec057050 con 0x7f26ec09a280 2020-11-20T13:56:36.775 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.775+0000 7f26e8ff9700 1 -- 172.21.15.40:0/1807810262 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16347 (secure 0 0 0) 0x7f26dc018020 con 0x7f26ec09a280 2020-11-20T13:56:36.775 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:56:36.775 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":24,"fsid":"da1ed737-5174-430f-a3c9-56f6bbfad13c","created":"2020-11-20T13:56:04.990651+0000","modified":"2020-11-20T13:56:25.428032+0000","last_up_change":"2020-11-20T13:56:13.893026+0000","last_in_change":"2020-11-20T13:56:13.893026+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T13:56:08.330017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T13:56:17.939684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T13:56:20.563740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T13:56:23.062623+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9ea4c667-8938-46d6-a5ca-092d55c1f79e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":22825},{"type":"v1","addr":"172.21.15.40:6818","nonce":22825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":22825},{"type":"v1","addr":"172.21.15.40:6822","nonce":22825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":22825},{"type":"v1","addr":"172.21.15.40:6830","nonce":22825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":22825},{"type":"v1","addr":"172.21.15.40:6826","nonce":22825}]},"public_addr":"172.21.15.40:6818/22825","cluster_addr":"172.21.15.40:6822/22825","heartbeat_back_addr":"172.21.15.40:6830/22825","heartbeat_front_addr":"172.21.15.40:6826/22825","state":["exists","up"]},{"osd":1,"uuid":"515708d9-5087-4e55-8d14-cdea1c00eacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":22823},{"type":"v1","addr":"172.21.15.40:6801","nonce":22823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":22823},{"type":"v1","addr":"172.21.15.40:6803","nonce":22823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":22823},{"type":"v1","addr":"172.21.15.40:6807","nonce":22823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":22823},{"type":"v1","addr":"172.21.15.40:6805","nonce":22823}]},"public_addr":"172.21.15.40:6801/22823","cluster_addr":"172.21.15.40:6803/22823","heartbeat_back_addr":"172.21.15.40:6807/22823","heartbeat_front_addr":"172.21.15.40:6805/22823","state":["exists","up"]},{"osd":2,"uuid":"4d86bca7-743e-4983-b35f-36676c405b36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":22824},{"type":"v1","addr":"172.21.15.40:6809","nonce":22824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":22824},{"type":"v1","addr":"172.21.15.40:6811","nonce":22824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":22824},{"type":"v1","addr":"172.21.15.40:6815","nonce":22824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":22824},{"type":"v1","addr":"172.21.15.40:6813","nonce":22824}]},"public_addr":"172.21.15.40:6809/22824","cluster_addr":"172.21.15.40:6811/22824","heartbeat_back_addr":"172.21.15.40:6815/22824","heartbeat_front_addr":"172.21.15.40:6813/22824","state":["exists","up"]},{"osd":3,"uuid":"f4dc726b-6b34-4e8b-8eeb-87060b58002d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":22826},{"type":"v1","addr":"172.21.15.40:6819","nonce":22826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":22826},{"type":"v1","addr":"172.21.15.40:6823","nonce":22826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":22826},{"type":"v1","addr":"172.21.15.40:6831","nonce":22826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":22826},{"type":"v1","addr":"172.21.15.40:6827","nonce":22826}]},"public_addr":"172.21.15.40:6819/22826","cluster_addr":"172.21.15.40:6823/22826","heartbeat_back_addr":"172.21.15.40:6831/22826","heartbeat_front_addr":"172.21.15.40:6827/22826","state":["exists","up"]},{"osd":4,"uuid":"25a6d53b-d1f0-4498-b7d5-ffa38661d403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":23181},{"type":"v1","addr":"172.21.15.116:6819","nonce":23181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":23181},{"type":"v1","addr":"172.21.15.116:6821","nonce":23181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":23181},{"type":"v1","addr":"172.21.15.116:6825","nonce":23181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":23181},{"type":"v1","addr":"172.21.15.116:6823","nonce":23181}]},"public_addr":"172.21.15.116:6819/23181","cluster_addr":"172.21.15.116:6821/23181","heartbeat_back_addr":"172.21.15.116:6825/23181","heartbeat_front_addr":"172.21.15.116:6823/23181","state":["exists","up"]},{"osd":5,"uuid":"7d7a5af8-229b-4351-9a21-a3d509cd4635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":23171},{"type":"v1","addr":"172.21.15.116:6803","nonce":23171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":23171},{"type":"v1","addr":"172.21.15.116:6805","nonce":23171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":23171},{"type":"v1","addr":"172.21.15.116:6809","nonce":23171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":23171},{"type":"v1","addr":"172.21.15.116:6807","nonce":23171}]},"public_addr":"172.21.15.116:6803/23171","cluster_addr":"172.21.15.116:6805/23171","heartbeat_back_addr":"172.21.15.116:6809/23171","heartbeat_front_addr":"172.21.15.116:6807/23171","state":["exists","up"]},{"osd":6,"uuid":"8247d248-a140-4e36-9796-9847a97df58d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":23172},{"type":"v1","addr":"172.21.15.116:6811","nonce":23172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":23172},{"type":"v1","addr":"172.21.15.116:6813","nonce":23172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":23172},{"type":"v1","addr":"172.21.15.116:6817","nonce":23172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":23172},{"type":"v1","addr":"172.21.15.116:6815","nonce":23172}]},"public_addr":"172.21.15.116:6811/23172","cluster_addr":"172.21.15.116:6813/23172","heartbeat_back_addr":"172.21.15.116:6817/23172","heartbeat_front_addr":"172.21.15.116:6815/23172","state":["exists","up"]},{"osd":7,"uuid":"472803c4-70b8-499d-b18a-71aaeec6aab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":23196},{"type":"v1","addr":"172.21.15.116:6827","nonce":23196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":23196},{"type":"v1","addr":"172.21.15.116:6829","nonce":23196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":23196},{"type":"v1","addr":"172.21.15.116:6833","nonce":23196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":23196},{"type":"v1","addr":"172.21.15.116:6831","nonce":23196}]},"public_addr":"172.21.15.116:6827/23196","cluster_addr":"172.21.15.116:6829/23196","heartbeat_back_addr":"172.21.15.116:6833/23196","heartbeat_front_addr":"172.21.15.116:6831/23196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.029357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:11.989797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.091097+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.014766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.070860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.071864+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.063400+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.109028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-20T13:56:36.776 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.776+0000 7f26f1aa1700 1 -- 172.21.15.40:0/1807810262 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f26cc051320 msgr2=0x7f26cc0537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:36.777 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.776+0000 7f26f1aa1700 1 --2- 172.21.15.40:0/1807810262 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f26cc051320 0x7f26cc0537e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f26e000c000 tx=0x7f26e0008040).stop 2020-11-20T13:56:36.777 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.776+0000 7f26f1aa1700 1 -- 172.21.15.40:0/1807810262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26ec09a280 msgr2=0x7f26ec060b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:36.778 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.776+0000 7f26f1aa1700 1 --2- 172.21.15.40:0/1807810262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26ec09a280 0x7f26ec060b40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f26ec0b73b0 tx=0x7f26dc007b50).stop 2020-11-20T13:56:36.778 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.776+0000 7f26f1aa1700 1 -- 172.21.15.40:0/1807810262 shutdown_connections 2020-11-20T13:56:36.779 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.776+0000 7f26f1aa1700 1 --2- 172.21.15.40:0/1807810262 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f26cc051320 0x7f26cc0537e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.779 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.776+0000 7f26f1aa1700 1 --2- 172.21.15.40:0/1807810262 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26ec0b88c0 0x7f26ec13cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.779 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.776+0000 7f26f1aa1700 1 --2- 172.21.15.40:0/1807810262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26ec09a280 0x7f26ec060b40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.779 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.776+0000 7f26f1aa1700 1 --2- 172.21.15.40:0/1807810262 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f26ec09bc40 0x7f26ec154720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.780 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.776+0000 7f26f1aa1700 1 -- 172.21.15.40:0/1807810262 >> 172.21.15.40:0/1807810262 conn(0x7f26ec0ba380 msgr2=0x7f26ec13dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:36.780 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.776+0000 7f26f1aa1700 1 -- 172.21.15.40:0/1807810262 shutdown_connections 2020-11-20T13:56:36.780 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.776+0000 7f26f1aa1700 1 -- 172.21.15.40:0/1807810262 wait complete. 2020-11-20T13:56:36.787 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T13:56:08.330017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T13:56:17.939684+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T13:56:20.563740+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T13:56:23.062623+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T13:56:36.788 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-20T13:56:36.907 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.907+0000 7fd29fbc1700 1 Processor -- start 2020-11-20T13:56:36.907 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.907+0000 7fd29fbc1700 1 -- start start 2020-11-20T13:56:36.909 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.908+0000 7fd29fbc1700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd2980b6c80 0x7fd2980b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:36.909 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.908+0000 7fd29fbc1700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2980b0d10 con 0x7fd298144bd0 2020-11-20T13:56:36.909 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.908+0000 7fd29fbc1700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2980b0fb0 con 0x7fd2980b1450 2020-11-20T13:56:36.910 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.908+0000 7fd29fbc1700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd298145060 con 0x7fd2980b6c80 2020-11-20T13:56:36.910 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.908+0000 7fd29d15c700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd298144bd0 0x7fd2980b40f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45590/0 (socket says 172.21.15.40:45590) 2020-11-20T13:56:36.910 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.908+0000 7fd29d15c700 1 -- 172.21.15.40:0/3727830106 learned_addr learned my addr 172.21.15.40:0/3727830106 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:36.910 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.908+0000 7fd29d95d700 1 --2- 172.21.15.40:0/3727830106 >> v2:172.21.15.116:3300/0 conn(0x7fd2980b6c80 0x7fd2980b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:36.911 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.908+0000 7fd29c95b700 1 -- 172.21.15.40:0/3727830106 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3053420692 0 0) 0x7fd2980b0d10 con 0x7fd298144bd0 2020-11-20T13:56:36.911 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.908+0000 7fd29c95b700 1 -- 172.21.15.40:0/3727830106 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd280003230 con 0x7fd298144bd0 2020-11-20T13:56:36.911 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.908+0000 7fd29d95d700 1 -- 172.21.15.40:0/3727830106 >> v1:172.21.15.116:6789/0 conn(0x7fd2980b1450 legacy=0x7fd29812ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:36.911 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.908+0000 7fd29d95d700 1 -- 172.21.15.40:0/3727830106 >> v1:172.21.15.40:6789/0 conn(0x7fd298144bd0 legacy=0x7fd2980b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:36.911 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.908+0000 7fd29c95b700 1 -- 172.21.15.40:0/3727830106 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2667405198 0 0) 0x7fd2980b0fb0 con 0x7fd2980b1450 2020-11-20T13:56:36.912 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.908+0000 7fd29d95d700 1 -- 172.21.15.40:0/3727830106 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2980b7060 con 0x7fd2980b6c80 2020-11-20T13:56:36.912 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.909+0000 7fd29d95d700 1 --2- 172.21.15.40:0/3727830106 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2980b6c80 0x7fd2980b31f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fd294001870 tx=0x7fd28c006d60).ready entity=mon.1 client_cookie=3b25fcbc09500250 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:36.912 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.909+0000 7fd29c95b700 1 -- 172.21.15.40:0/3727830106 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2940016d0 con 0x7fd2980b6c80 2020-11-20T13:56:36.912 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.909+0000 7fd29c95b700 1 -- 172.21.15.40:0/3727830106 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd28c009670 con 0x7fd2980b6c80 2020-11-20T13:56:36.912 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.909+0000 7fd29c95b700 1 -- 172.21.15.40:0/3727830106 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd28c0097e0 con 0x7fd2980b6c80 2020-11-20T13:56:36.913 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.909+0000 7fd29fbc1700 1 -- 172.21.15.40:0/3727830106 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2980b6c80 msgr2=0x7fd2980b31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:36.913 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.909+0000 7fd29fbc1700 1 --2- 172.21.15.40:0/3727830106 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2980b6c80 0x7fd2980b31f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fd294001870 tx=0x7fd28c006d60).stop 2020-11-20T13:56:36.916 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.909+0000 7fd29fbc1700 1 -- 172.21.15.40:0/3727830106 shutdown_connections 2020-11-20T13:56:36.916 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.909+0000 7fd29fbc1700 1 --2- 172.21.15.40:0/3727830106 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2980b6c80 0x7fd2980b31f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.916 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.909+0000 7fd29fbc1700 1 -- 172.21.15.40:0/3727830106 >> 172.21.15.40:0/3727830106 conn(0x7fd2980ba1d0 msgr2=0x7fd2980b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:36.917 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.909+0000 7fd29fbc1700 1 -- 172.21.15.40:0/3727830106 shutdown_connections 2020-11-20T13:56:36.917 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.909+0000 7fd29fbc1700 1 -- 172.21.15.40:0/3727830106 wait complete. 2020-11-20T13:56:36.917 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.910+0000 7fd29fbc1700 1 Processor -- start 2020-11-20T13:56:36.917 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.910+0000 7fd29fbc1700 1 -- start start 2020-11-20T13:56:36.918 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.911+0000 7fd29fbc1700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2980b1450 0x7fd298066970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:36.918 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.911+0000 7fd29fbc1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2980b6c80 0x7fd298147570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:36.918 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.911+0000 7fd29fbc1700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd298144bd0 0x7fd298154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:36.918 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.911+0000 7fd29fbc1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd298065f90 con 0x7fd2980b6c80 2020-11-20T13:56:36.918 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.911+0000 7fd29fbc1700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd298066230 con 0x7fd2980b1450 2020-11-20T13:56:36.919 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.911+0000 7fd29fbc1700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2980664d0 con 0x7fd298144bd0 2020-11-20T13:56:36.919 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.911+0000 7fd29d15c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2980b6c80 0x7fd298147570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:36.919 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.911+0000 7fd29d15c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2980b6c80 0x7fd298147570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55002/0 (socket says 172.21.15.40:55002) 2020-11-20T13:56:36.919 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.911+0000 7fd29d15c700 1 -- 172.21.15.40:0/665835000 learned_addr learned my addr 172.21.15.40:0/665835000 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:36.919 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.911+0000 7fd29d95d700 1 --2- 172.21.15.40:0/665835000 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2980b1450 0x7fd298066970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:36.920 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.911+0000 7fd29e15e700 1 --2- 172.21.15.40:0/665835000 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd298144bd0 0x7fd298154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:36.920 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.911+0000 7fd29d15c700 1 -- 172.21.15.40:0/665835000 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd298144bd0 msgr2=0x7fd298154110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:36.920 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.911+0000 7fd29d15c700 1 --2- 172.21.15.40:0/665835000 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd298144bd0 0x7fd298154110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.920 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.911+0000 7fd29d15c700 1 -- 172.21.15.40:0/665835000 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2980b1450 msgr2=0x7fd298066970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:36.920 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.911+0000 7fd29d15c700 1 --2- 172.21.15.40:0/665835000 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2980b1450 0x7fd298066970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.921 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.911+0000 7fd29d15c700 1 -- 172.21.15.40:0/665835000 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd298066e60 con 0x7fd2980b6c80 2020-11-20T13:56:36.921 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.911+0000 7fd29e15e700 1 --2- 172.21.15.40:0/665835000 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd298144bd0 0x7fd298154110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:36.921 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.911+0000 7fd29d15c700 1 --2- 172.21.15.40:0/665835000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2980b6c80 0x7fd298147570 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7fd2940018b0 tx=0x7fd294001a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:36.921 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.912+0000 7fd28affd700 1 -- 172.21.15.40:0/665835000 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd294007170 con 0x7fd2980b6c80 2020-11-20T13:56:36.922 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.912+0000 7fd28affd700 1 -- 172.21.15.40:0/665835000 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2940072d0 con 0x7fd2980b6c80 2020-11-20T13:56:36.922 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.912+0000 7fd29fbc1700 1 -- 172.21.15.40:0/665835000 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd298065bf0 con 0x7fd2980b6c80 2020-11-20T13:56:36.922 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.912+0000 7fd28affd700 1 -- 172.21.15.40:0/665835000 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd29400a960 con 0x7fd2980b6c80 2020-11-20T13:56:36.922 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.912+0000 7fd29fbc1700 1 -- 172.21.15.40:0/665835000 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd298065a00 con 0x7fd2980b6c80 2020-11-20T13:56:36.922 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.913+0000 7fd28affd700 1 -- 172.21.15.40:0/665835000 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fd29401c030 con 0x7fd2980b6c80 2020-11-20T13:56:36.922 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.913+0000 7fd29fbc1700 1 -- 172.21.15.40:0/665835000 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd298147de0 con 0x7fd2980b6c80 2020-11-20T13:56:36.923 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.913+0000 7fd28affd700 1 --2- 172.21.15.40:0/665835000 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fd2780513c0 0x7fd278053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:36.923 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.913+0000 7fd28affd700 1 -- 172.21.15.40:0/665835000 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fd294069200 con 0x7fd2980b6c80 2020-11-20T13:56:36.923 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.913+0000 7fd29d95d700 1 --2- 172.21.15.40:0/665835000 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fd2780513c0 0x7fd278053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:36.923 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.913+0000 7fd29d95d700 1 --2- 172.21.15.40:0/665835000 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fd2780513c0 0x7fd278053880 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fd28c017000 tx=0x7fd2980b0ea0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:36.924 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:36.916+0000 7fd28affd700 1 -- 172.21.15.40:0/665835000 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fd294041420 con 0x7fd2980b6c80 2020-11-20T13:56:37.077 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.076+0000 7fd29fbc1700 1 -- 172.21.15.40:0/665835000 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fd298057050 con 0x7fd2980b6c80 2020-11-20T13:56:37.078 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.077+0000 7fd28affd700 1 -- 172.21.15.40:0/665835000 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v24) v1 ==== 110+0+10 (secure 0 0 0) 0x7fd294044a40 con 0x7fd2980b6c80 2020-11-20T13:56:37.078 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2020-11-20T13:56:37.079 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.078+0000 7fd29fbc1700 1 -- 172.21.15.40:0/665835000 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fd2780513c0 msgr2=0x7fd278053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:37.079 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.078+0000 7fd29fbc1700 1 --2- 172.21.15.40:0/665835000 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fd2780513c0 0x7fd278053880 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fd28c017000 tx=0x7fd2980b0ea0).stop 2020-11-20T13:56:37.080 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.078+0000 7fd29fbc1700 1 -- 172.21.15.40:0/665835000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2980b6c80 msgr2=0x7fd298147570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:37.080 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.078+0000 7fd29fbc1700 1 --2- 172.21.15.40:0/665835000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2980b6c80 0x7fd298147570 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7fd2940018b0 tx=0x7fd294001a40).stop 2020-11-20T13:56:37.080 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.078+0000 7fd29fbc1700 1 -- 172.21.15.40:0/665835000 shutdown_connections 2020-11-20T13:56:37.080 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.078+0000 7fd29fbc1700 1 --2- 172.21.15.40:0/665835000 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fd2780513c0 0x7fd278053880 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.081 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.078+0000 7fd29fbc1700 1 --2- 172.21.15.40:0/665835000 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd298144bd0 0x7fd298154110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.081 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.078+0000 7fd29fbc1700 1 --2- 172.21.15.40:0/665835000 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2980b1450 0x7fd298066970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.082 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.078+0000 7fd29fbc1700 1 --2- 172.21.15.40:0/665835000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2980b6c80 0x7fd298147570 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.082 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.078+0000 7fd29fbc1700 1 -- 172.21.15.40:0/665835000 >> 172.21.15.40:0/665835000 conn(0x7fd2980ba1d0 msgr2=0x7fd2980b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:37.082 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.079+0000 7fd29fbc1700 1 -- 172.21.15.40:0/665835000 shutdown_connections 2020-11-20T13:56:37.082 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.079+0000 7fd29fbc1700 1 -- 172.21.15.40:0/665835000 wait complete. 2020-11-20T13:56:37.092 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-20T13:56:37.211 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.210+0000 7f3bd2f2a700 1 Processor -- start 2020-11-20T13:56:37.211 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.210+0000 7f3bd2f2a700 1 -- start start 2020-11-20T13:56:37.214 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.211+0000 7f3bd2f2a700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3bcc061f90 0x7f3bcc14b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:37.215 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.211+0000 7f3bd2f2a700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bcc061310 con 0x7f3bcc0b6980 2020-11-20T13:56:37.215 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.211+0000 7f3bd2f2a700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bcc0615b0 con 0x7f3bcc05c510 2020-11-20T13:56:37.215 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.211+0000 7f3bd2f2a700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc061850 con 0x7f3bcc061f90 2020-11-20T13:56:37.215 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.211+0000 7f3bd14c7700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3bcc0b6980 0x7f3bcc0b6df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45608/0 (socket says 172.21.15.40:45608) 2020-11-20T13:56:37.216 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.211+0000 7f3bd14c7700 1 -- 172.21.15.40:0/4111303992 learned_addr learned my addr 172.21.15.40:0/4111303992 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:37.216 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.212+0000 7f3bcbfff700 1 --2- 172.21.15.40:0/4111303992 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3bcc061f90 0x7f3bcc14b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:37.216 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.212+0000 7f3bcb7fe700 1 -- 172.21.15.40:0/4111303992 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2167248485 0 0) 0x7f3bcc061310 con 0x7f3bcc0b6980 2020-11-20T13:56:37.216 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.212+0000 7f3bcb7fe700 1 -- 172.21.15.40:0/4111303992 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bb8003230 con 0x7f3bcc0b6980 2020-11-20T13:56:37.216 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.212+0000 7f3bcb7fe700 1 -- 172.21.15.40:0/4111303992 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3202266291 0 0) 0x7f3bb8003230 con 0x7f3bcc0b6980 2020-11-20T13:56:37.217 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.212+0000 7f3bcb7fe700 1 -- 172.21.15.40:0/4111303992 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3bcc061f90 msgr2=0x7f3bcc14b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:37.217 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.212+0000 7f3bcb7fe700 1 --2- 172.21.15.40:0/4111303992 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3bcc061f90 0x7f3bcc14b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.217 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.212+0000 7f3bcb7fe700 1 -- 172.21.15.40:0/4111303992 >> v1:172.21.15.116:6789/0 conn(0x7f3bcc05c510 legacy=0x7f3bcc12d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:37.217 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.212+0000 7f3bcb7fe700 1 -- 172.21.15.40:0/4111303992 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bcc061af0 con 0x7f3bcc0b6980 2020-11-20T13:56:37.218 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.212+0000 7f3bcb7fe700 1 -- 172.21.15.40:0/4111303992 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2366654341 0 0) 0x7f3bc4001a50 con 0x7f3bcc0b6980 2020-11-20T13:56:37.218 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.212+0000 7f3bcb7fe700 1 -- 172.21.15.40:0/4111303992 >> v1:172.21.15.40:6789/0 conn(0x7f3bcc0b6980 legacy=0x7f3bcc0b6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:37.218 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.212+0000 7f3bcb7fe700 1 --2- 172.21.15.40:0/4111303992 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3bb8003750 0x7f3bb8003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:37.218 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.212+0000 7f3bcb7fe700 1 --2- 172.21.15.40:0/4111303992 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3bb8004090 0x7f3bb8004570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:37.218 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.212+0000 7f3bcb7fe700 1 --2- 172.21.15.40:0/4111303992 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3bcc061f90 0x7f3bb8004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:37.219 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.212+0000 7f3bcb7fe700 1 -- 172.21.15.40:0/4111303992 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc061850 con 0x7f3bb8004090 2020-11-20T13:56:37.219 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.212+0000 7f3bcb7fe700 1 -- 172.21.15.40:0/4111303992 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb8005140 con 0x7f3bb8003750 2020-11-20T13:56:37.219 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.212+0000 7f3bcb7fe700 1 -- 172.21.15.40:0/4111303992 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb8005300 con 0x7f3bcc061f90 2020-11-20T13:56:37.219 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.212+0000 7f3bcbfff700 1 --2- 172.21.15.40:0/4111303992 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3bb8004090 0x7f3bb8004570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:37.219 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.212+0000 7f3bd0cc6700 1 --2- 172.21.15.40:0/4111303992 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3bb8003750 0x7f3bb8003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:37.220 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.212+0000 7f3bd14c7700 1 --2- 172.21.15.40:0/4111303992 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3bcc061f90 0x7f3bb8004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:37.220 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.212+0000 7f3bcbfff700 1 -- 172.21.15.40:0/4111303992 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3bcc061f90 msgr2=0x7f3bb8004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:37.220 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.212+0000 7f3bcbfff700 1 --2- 172.21.15.40:0/4111303992 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3bcc061f90 0x7f3bb8004bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.220 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.212+0000 7f3bcbfff700 1 -- 172.21.15.40:0/4111303992 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3bb8003750 msgr2=0x7f3bb8003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:37.220 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.212+0000 7f3bcbfff700 1 --2- 172.21.15.40:0/4111303992 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3bb8003750 0x7f3bb8003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.221 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.212+0000 7f3bcbfff700 1 -- 172.21.15.40:0/4111303992 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb8003230 con 0x7f3bb8004090 2020-11-20T13:56:37.221 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.212+0000 7f3bd0cc6700 1 --2- 172.21.15.40:0/4111303992 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3bb8003750 0x7f3bb8003b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:56:37.221 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.212+0000 7f3bcbfff700 1 --2- 172.21.15.40:0/4111303992 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3bb8004090 0x7f3bb8004570 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f3bc000c000 tx=0x7f3bc0008040).ready entity=mon.0 client_cookie=967ba503c531250f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:37.221 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.213+0000 7f3bcb7fe700 1 -- 172.21.15.40:0/4111303992 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3bc000e8a0 con 0x7f3bb8004090 2020-11-20T13:56:37.221 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.213+0000 7f3bcb7fe700 1 -- 172.21.15.40:0/4111303992 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc000ea00 con 0x7f3bb8004090 2020-11-20T13:56:37.222 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.213+0000 7f3bcb7fe700 1 -- 172.21.15.40:0/4111303992 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3bc000ed60 con 0x7f3bb8004090 2020-11-20T13:56:37.222 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.213+0000 7f3bd2f2a700 1 -- 172.21.15.40:0/4111303992 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3bb8004090 msgr2=0x7f3bb8004570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:37.222 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.213+0000 7f3bd2f2a700 1 --2- 172.21.15.40:0/4111303992 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3bb8004090 0x7f3bb8004570 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f3bc000c000 tx=0x7f3bc0008040).stop 2020-11-20T13:56:37.222 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.213+0000 7f3bd14c7700 1 -- 172.21.15.40:0/4111303992 reap_dead start 2020-11-20T13:56:37.222 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.213+0000 7f3bd2f2a700 1 -- 172.21.15.40:0/4111303992 shutdown_connections 2020-11-20T13:56:37.223 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.213+0000 7f3bd2f2a700 1 -- 172.21.15.40:0/4111303992 >> 172.21.15.40:0/4111303992 conn(0x7f3bcc0bdb70 msgr2=0x7f3bcc0bdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:37.223 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.213+0000 7f3bd2f2a700 1 -- 172.21.15.40:0/4111303992 shutdown_connections 2020-11-20T13:56:37.223 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.213+0000 7f3bd2f2a700 1 -- 172.21.15.40:0/4111303992 wait complete. 2020-11-20T13:56:37.223 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.214+0000 7f3bd2f2a700 1 Processor -- start 2020-11-20T13:56:37.223 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.214+0000 7f3bd2f2a700 1 -- start start 2020-11-20T13:56:37.226 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.215+0000 7f3bd2f2a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3bcc148920 0x7f3bcc0accf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:37.226 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.215+0000 7f3bd2f2a700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3bcc15a240 0x7f3bcc0b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:37.226 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.215+0000 7f3bd2f2a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3bcc1436a0 0x7f3bcc143b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:37.227 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.215+0000 7f3bd2f2a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc0b45d0 con 0x7f3bcc148920 2020-11-20T13:56:37.227 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.215+0000 7f3bd2f2a700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc0b4860 con 0x7f3bcc1436a0 2020-11-20T13:56:37.227 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.215+0000 7f3bd2f2a700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc0b4b00 con 0x7f3bcc15a240 2020-11-20T13:56:37.227 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.215+0000 7f3bd0cc6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3bcc148920 0x7f3bcc0accf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:37.228 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.215+0000 7f3bd0cc6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3bcc148920 0x7f3bcc0accf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55020/0 (socket says 172.21.15.40:55020) 2020-11-20T13:56:37.228 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.215+0000 7f3bd0cc6700 1 -- 172.21.15.40:0/2540057577 learned_addr learned my addr 172.21.15.40:0/2540057577 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:37.228 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.215+0000 7f3bcbfff700 1 --2- 172.21.15.40:0/2540057577 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3bcc15a240 0x7f3bcc0b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:37.228 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.215+0000 7f3bd14c7700 1 --2- 172.21.15.40:0/2540057577 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3bcc1436a0 0x7f3bcc143b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:37.228 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.215+0000 7f3bd0cc6700 1 -- 172.21.15.40:0/2540057577 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3bcc15a240 msgr2=0x7f3bcc0b40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:37.229 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.215+0000 7f3bd0cc6700 1 --2- 172.21.15.40:0/2540057577 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3bcc15a240 0x7f3bcc0b40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.229 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.215+0000 7f3bd0cc6700 1 -- 172.21.15.40:0/2540057577 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3bcc1436a0 msgr2=0x7f3bcc143b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:37.229 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.215+0000 7f3bd0cc6700 1 --2- 172.21.15.40:0/2540057577 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3bcc1436a0 0x7f3bcc143b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.229 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.215+0000 7f3bd0cc6700 1 -- 172.21.15.40:0/2540057577 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bcc148d00 con 0x7f3bcc148920 2020-11-20T13:56:37.230 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.215+0000 7f3bcbfff700 1 --2- 172.21.15.40:0/2540057577 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3bcc15a240 0x7f3bcc0b40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:37.230 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.215+0000 7f3bd14c7700 1 --2- 172.21.15.40:0/2540057577 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3bcc1436a0 0x7f3bcc143b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:37.230 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.215+0000 7f3bd0cc6700 1 --2- 172.21.15.40:0/2540057577 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3bcc148920 0x7f3bcc0accf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f3bbc00d000 tx=0x7f3bbc009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:37.230 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.216+0000 7f3bc9ffb700 1 -- 172.21.15.40:0/2540057577 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3bbc00da10 con 0x7f3bcc148920 2020-11-20T13:56:37.230 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.216+0000 7f3bc9ffb700 1 -- 172.21.15.40:0/2540057577 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bbc00db70 con 0x7f3bcc148920 2020-11-20T13:56:37.231 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.216+0000 7f3bc9ffb700 1 -- 172.21.15.40:0/2540057577 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3bbc0133f0 con 0x7f3bcc148920 2020-11-20T13:56:37.231 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.216+0000 7f3bd2f2a700 1 -- 172.21.15.40:0/2540057577 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bcc0b0990 con 0x7f3bcc148920 2020-11-20T13:56:37.231 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.216+0000 7f3bd2f2a700 1 -- 172.21.15.40:0/2540057577 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bcc0b0e60 con 0x7f3bcc148920 2020-11-20T13:56:37.231 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.217+0000 7f3bd2f2a700 1 -- 172.21.15.40:0/2540057577 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bcc057050 con 0x7f3bcc148920 2020-11-20T13:56:37.231 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.217+0000 7f3bc9ffb700 1 -- 172.21.15.40:0/2540057577 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f3bbc01a030 con 0x7f3bcc148920 2020-11-20T13:56:37.232 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.217+0000 7f3bc9ffb700 1 --2- 172.21.15.40:0/2540057577 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f3bac051290 0x7f3bac053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:37.232 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.217+0000 7f3bc9ffb700 1 -- 172.21.15.40:0/2540057577 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f3bbc068ef0 con 0x7f3bcc148920 2020-11-20T13:56:37.232 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.217+0000 7f3bcbfff700 1 --2- 172.21.15.40:0/2540057577 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f3bac051290 0x7f3bac053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:37.232 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.217+0000 7f3bcbfff700 1 --2- 172.21.15.40:0/2540057577 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f3bac051290 0x7f3bac053750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f3bc000c000 tx=0x7f3bc001a000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:37.232 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.220+0000 7f3bc9ffb700 1 -- 172.21.15.40:0/2540057577 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f3bbc041110 con 0x7f3bcc148920 2020-11-20T13:56:37.381 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.381+0000 7f3bd2f2a700 1 -- 172.21.15.40:0/2540057577 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3bcc147300 con 0x7f3bcc148920 2020-11-20T13:56:37.382 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.381+0000 7f3bc9ffb700 1 -- 172.21.15.40:0/2540057577 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3bbc044730 con 0x7f3bcc148920 2020-11-20T13:56:37.382 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2020-11-20T13:56:37.383 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.383+0000 7f3bd2f2a700 1 -- 172.21.15.40:0/2540057577 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f3bac051290 msgr2=0x7f3bac053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:37.383 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.383+0000 7f3bd2f2a700 1 --2- 172.21.15.40:0/2540057577 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f3bac051290 0x7f3bac053750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f3bc000c000 tx=0x7f3bc001a000).stop 2020-11-20T13:56:37.384 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.383+0000 7f3bd2f2a700 1 -- 172.21.15.40:0/2540057577 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3bcc148920 msgr2=0x7f3bcc0accf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:37.384 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.383+0000 7f3bd2f2a700 1 --2- 172.21.15.40:0/2540057577 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3bcc148920 0x7f3bcc0accf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f3bbc00d000 tx=0x7f3bbc009040).stop 2020-11-20T13:56:37.384 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.383+0000 7f3bd2f2a700 1 -- 172.21.15.40:0/2540057577 shutdown_connections 2020-11-20T13:56:37.384 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.383+0000 7f3bd2f2a700 1 --2- 172.21.15.40:0/2540057577 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f3bac051290 0x7f3bac053750 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.385 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.383+0000 7f3bd2f2a700 1 --2- 172.21.15.40:0/2540057577 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3bcc1436a0 0x7f3bcc143b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.385 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.383+0000 7f3bd2f2a700 1 --2- 172.21.15.40:0/2540057577 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3bcc148920 0x7f3bcc0accf0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.385 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.383+0000 7f3bd2f2a700 1 --2- 172.21.15.40:0/2540057577 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3bcc15a240 0x7f3bcc0b40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.385 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.383+0000 7f3bd2f2a700 1 -- 172.21.15.40:0/2540057577 >> 172.21.15.40:0/2540057577 conn(0x7f3bcc0bdb70 msgr2=0x7f3bcc0bd330 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:37.385 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.383+0000 7f3bd2f2a700 1 -- 172.21.15.40:0/2540057577 shutdown_connections 2020-11-20T13:56:37.386 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.383+0000 7f3bd2f2a700 1 -- 172.21.15.40:0/2540057577 wait complete. 2020-11-20T13:56:37.393 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-20T13:56:37.510 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.510+0000 7f2ca8a09700 1 Processor -- start 2020-11-20T13:56:37.511 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.510+0000 7f2ca8a09700 1 -- start start 2020-11-20T13:56:37.513 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.511+0000 7f2ca8a09700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2ca40b5bf0 0x7f2ca405f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:37.513 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.511+0000 7f2ca8a09700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ca4144390 0x7f2ca405f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:37.513 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.511+0000 7f2ca8a09700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ca40b5fd0 con 0x7f2ca405fd70 2020-11-20T13:56:37.513 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.511+0000 7f2ca8a09700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ca40b05f0 con 0x7f2ca40b5bf0 2020-11-20T13:56:37.514 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.511+0000 7f2ca8a09700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca4144870 con 0x7f2ca4144390 2020-11-20T13:56:37.514 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.511+0000 7f2ca2d9d700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2ca405fd70 0x7f2ca40b2950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45624/0 (socket says 172.21.15.40:45624) 2020-11-20T13:56:37.514 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.511+0000 7f2ca2d9d700 1 -- 172.21.15.40:0/3427704604 learned_addr learned my addr 172.21.15.40:0/3427704604 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:37.514 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.511+0000 7f2ca1d9b700 1 --2- 172.21.15.40:0/3427704604 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ca4144390 0x7f2ca405f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:37.514 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.511+0000 7f2ca259c700 1 --2- 172.21.15.40:0/3427704604 >> v2:172.21.15.116:3300/0 conn(0x7f2ca40b5bf0 0x7f2ca405f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:37.515 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.511+0000 7f2ca159a700 1 -- 172.21.15.40:0/3427704604 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3993195460 0 0) 0x7f2ca40b5fd0 con 0x7f2ca405fd70 2020-11-20T13:56:37.515 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.511+0000 7f2ca159a700 1 -- 172.21.15.40:0/3427704604 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c88003230 con 0x7f2ca405fd70 2020-11-20T13:56:37.515 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.512+0000 7f2ca1d9b700 1 -- 172.21.15.40:0/3427704604 >> v2:172.21.15.116:3300/0 conn(0x7f2ca40b5bf0 msgr2=0x7f2ca405f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:37.515 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.512+0000 7f2ca1d9b700 1 --2- 172.21.15.40:0/3427704604 >> v2:172.21.15.116:3300/0 conn(0x7f2ca40b5bf0 0x7f2ca405f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.515 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.512+0000 7f2ca1d9b700 1 -- 172.21.15.40:0/3427704604 >> v1:172.21.15.40:6789/0 conn(0x7f2ca405fd70 legacy=0x7f2ca40b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:37.516 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.512+0000 7f2ca1d9b700 1 -- 172.21.15.40:0/3427704604 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca40b0c30 con 0x7f2ca4144390 2020-11-20T13:56:37.516 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.512+0000 7f2ca1d9b700 1 --2- 172.21.15.40:0/3427704604 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ca4144390 0x7f2ca405f880 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f2c98002f30 tx=0x7f2c98006dc0).ready entity=mon.2 client_cookie=f780927d562460ca server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:37.516 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.512+0000 7f2ca159a700 1 -- 172.21.15.40:0/3427704604 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c980096a0 con 0x7f2ca4144390 2020-11-20T13:56:37.516 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.512+0000 7f2ca159a700 1 -- 172.21.15.40:0/3427704604 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c98009800 con 0x7f2ca4144390 2020-11-20T13:56:37.516 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.512+0000 7f2ca159a700 1 -- 172.21.15.40:0/3427704604 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c98009b60 con 0x7f2ca4144390 2020-11-20T13:56:37.517 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.512+0000 7f2ca8a09700 1 -- 172.21.15.40:0/3427704604 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ca4144390 msgr2=0x7f2ca405f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:37.517 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.512+0000 7f2ca8a09700 1 --2- 172.21.15.40:0/3427704604 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ca4144390 0x7f2ca405f880 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f2c98002f30 tx=0x7f2c98006dc0).stop 2020-11-20T13:56:37.517 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.512+0000 7f2ca8a09700 1 -- 172.21.15.40:0/3427704604 shutdown_connections 2020-11-20T13:56:37.517 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.512+0000 7f2ca8a09700 1 --2- 172.21.15.40:0/3427704604 >> v2:172.21.15.116:3300/0 conn(0x7f2ca40b5bf0 0x7f2ca405f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.517 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.512+0000 7f2ca8a09700 1 --2- 172.21.15.40:0/3427704604 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ca4144390 0x7f2ca405f880 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.518 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.512+0000 7f2ca8a09700 1 -- 172.21.15.40:0/3427704604 >> 172.21.15.40:0/3427704604 conn(0x7f2ca40ba1d0 msgr2=0x7f2ca40b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:37.518 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.512+0000 7f2ca8a09700 1 -- 172.21.15.40:0/3427704604 shutdown_connections 2020-11-20T13:56:37.518 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.512+0000 7f2ca8a09700 1 -- 172.21.15.40:0/3427704604 wait complete. 2020-11-20T13:56:37.521 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.513+0000 7f2ca8a09700 1 Processor -- start 2020-11-20T13:56:37.521 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.513+0000 7f2ca8a09700 1 -- start start 2020-11-20T13:56:37.521 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.514+0000 7f2ca8a09700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ca405fd70 0x7f2ca4136520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:37.521 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.514+0000 7f2ca8a09700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ca40b5bf0 0x7f2ca413c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:37.522 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.514+0000 7f2ca8a09700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ca4144390 0x7f2ca40af480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:37.522 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.514+0000 7f2ca8a09700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca40afb90 con 0x7f2ca40b5bf0 2020-11-20T13:56:37.522 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.514+0000 7f2ca8a09700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca4134a30 con 0x7f2ca405fd70 2020-11-20T13:56:37.522 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.514+0000 7f2ca8a09700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca4134cd0 con 0x7f2ca4144390 2020-11-20T13:56:37.522 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.514+0000 7f2ca1d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ca40b5bf0 0x7f2ca413c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:37.523 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.514+0000 7f2ca1d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ca40b5bf0 0x7f2ca413c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55036/0 (socket says 172.21.15.40:55036) 2020-11-20T13:56:37.523 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.514+0000 7f2ca1d9b700 1 -- 172.21.15.40:0/1483974750 learned_addr learned my addr 172.21.15.40:0/1483974750 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:37.523 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.514+0000 7f2ca2d9d700 1 --2- 172.21.15.40:0/1483974750 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ca4144390 0x7f2ca40af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:37.523 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.514+0000 7f2ca259c700 1 --2- 172.21.15.40:0/1483974750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ca405fd70 0x7f2ca4136520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:37.523 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.515+0000 7f2ca1d9b700 1 -- 172.21.15.40:0/1483974750 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ca4144390 msgr2=0x7f2ca40af480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:37.524 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.515+0000 7f2ca1d9b700 1 --2- 172.21.15.40:0/1483974750 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ca4144390 0x7f2ca40af480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.524 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.515+0000 7f2ca1d9b700 1 -- 172.21.15.40:0/1483974750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ca405fd70 msgr2=0x7f2ca4136520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:37.524 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.515+0000 7f2ca1d9b700 1 --2- 172.21.15.40:0/1483974750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ca405fd70 0x7f2ca4136520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.524 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.515+0000 7f2ca1d9b700 1 -- 172.21.15.40:0/1483974750 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca4135170 con 0x7f2ca40b5bf0 2020-11-20T13:56:37.524 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.515+0000 7f2ca2d9d700 1 --2- 172.21.15.40:0/1483974750 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ca4144390 0x7f2ca40af480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:56:37.525 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.515+0000 7f2ca259c700 1 --2- 172.21.15.40:0/1483974750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ca405fd70 0x7f2ca4136520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:56:37.525 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.515+0000 7f2ca1d9b700 1 --2- 172.21.15.40:0/1483974750 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ca40b5bf0 0x7f2ca413c5c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f2c98002f30 tx=0x7f2c98015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:37.525 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.515+0000 7f2c937fe700 1 -- 172.21.15.40:0/1483974750 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c98009ea0 con 0x7f2ca40b5bf0 2020-11-20T13:56:37.525 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.515+0000 7f2c937fe700 1 -- 172.21.15.40:0/1483974750 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c98011b80 con 0x7f2ca40b5bf0 2020-11-20T13:56:37.525 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.515+0000 7f2ca8a09700 1 -- 172.21.15.40:0/1483974750 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ca4159550 con 0x7f2ca40b5bf0 2020-11-20T13:56:37.526 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.515+0000 7f2c937fe700 1 -- 172.21.15.40:0/1483974750 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c9801d450 con 0x7f2ca40b5bf0 2020-11-20T13:56:37.526 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.515+0000 7f2ca8a09700 1 -- 172.21.15.40:0/1483974750 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ca4132e50 con 0x7f2ca40b5bf0 2020-11-20T13:56:37.526 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.516+0000 7f2c937fe700 1 -- 172.21.15.40:0/1483974750 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f2c98024030 con 0x7f2ca40b5bf0 2020-11-20T13:56:37.526 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.516+0000 7f2ca8a09700 1 -- 172.21.15.40:0/1483974750 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ca4057050 con 0x7f2ca40b5bf0 2020-11-20T13:56:37.526 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.516+0000 7f2c937fe700 1 --2- 172.21.15.40:0/1483974750 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f2c800513c0 0x7f2c80053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:37.527 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.516+0000 7f2c937fe700 1 -- 172.21.15.40:0/1483974750 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f2c9806a250 con 0x7f2ca40b5bf0 2020-11-20T13:56:37.527 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.516+0000 7f2ca259c700 1 --2- 172.21.15.40:0/1483974750 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f2c800513c0 0x7f2c80053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:37.527 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.517+0000 7f2ca259c700 1 --2- 172.21.15.40:0/1483974750 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f2c800513c0 0x7f2c80053880 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f2c9c00b000 tx=0x7f2c9c0091a0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:37.527 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.520+0000 7f2c937fe700 1 -- 172.21.15.40:0/1483974750 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f2c980424b0 con 0x7f2ca40b5bf0 2020-11-20T13:56:37.679 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.679+0000 7f2ca8a09700 1 -- 172.21.15.40:0/1483974750 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2ca405edb0 con 0x7f2ca40b5bf0 2020-11-20T13:56:37.680 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.679+0000 7f2c937fe700 1 -- 172.21.15.40:0/1483974750 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+10 (secure 0 0 0) 0x7f2c980220a0 con 0x7f2ca40b5bf0 2020-11-20T13:56:37.680 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2020-11-20T13:56:37.681 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.681+0000 7f2ca8a09700 1 -- 172.21.15.40:0/1483974750 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f2c800513c0 msgr2=0x7f2c80053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:37.681 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.681+0000 7f2ca8a09700 1 --2- 172.21.15.40:0/1483974750 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f2c800513c0 0x7f2c80053880 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f2c9c00b000 tx=0x7f2c9c0091a0).stop 2020-11-20T13:56:37.682 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.681+0000 7f2ca8a09700 1 -- 172.21.15.40:0/1483974750 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ca40b5bf0 msgr2=0x7f2ca413c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:37.682 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.681+0000 7f2ca8a09700 1 --2- 172.21.15.40:0/1483974750 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ca40b5bf0 0x7f2ca413c5c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f2c98002f30 tx=0x7f2c98015000).stop 2020-11-20T13:56:37.683 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.681+0000 7f2ca8a09700 1 -- 172.21.15.40:0/1483974750 shutdown_connections 2020-11-20T13:56:37.683 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.681+0000 7f2ca8a09700 1 --2- 172.21.15.40:0/1483974750 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f2c800513c0 0x7f2c80053880 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.683 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.681+0000 7f2ca8a09700 1 --2- 172.21.15.40:0/1483974750 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ca4144390 0x7f2ca40af480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.683 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.681+0000 7f2ca8a09700 1 --2- 172.21.15.40:0/1483974750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ca405fd70 0x7f2ca4136520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.684 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.681+0000 7f2ca8a09700 1 --2- 172.21.15.40:0/1483974750 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ca40b5bf0 0x7f2ca413c5c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.684 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.681+0000 7f2ca8a09700 1 -- 172.21.15.40:0/1483974750 >> 172.21.15.40:0/1483974750 conn(0x7f2ca40ba1d0 msgr2=0x7f2ca40b7650 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:37.684 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.681+0000 7f2ca8a09700 1 -- 172.21.15.40:0/1483974750 shutdown_connections 2020-11-20T13:56:37.684 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.681+0000 7f2ca8a09700 1 -- 172.21.15.40:0/1483974750 wait complete. 2020-11-20T13:56:37.691 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-20T13:56:37.808 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.807+0000 7f0a833fb700 1 Processor -- start 2020-11-20T13:56:37.808 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.807+0000 7f0a833fb700 1 -- start start 2020-11-20T13:56:37.811 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.808+0000 7f0a833fb700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0a7c0b6c80 0x7f0a7c0b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:37.811 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.808+0000 7f0a833fb700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0a7c144bd0 0x7f0a7c0b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:37.812 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.808+0000 7f0a833fb700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a7c0b0d10 con 0x7f0a7c0b45e0 2020-11-20T13:56:37.812 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.808+0000 7f0a833fb700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a7c145030 con 0x7f0a7c144bd0 2020-11-20T13:56:37.812 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.808+0000 7f0a833fb700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c0b0fb0 con 0x7f0a7c0b6c80 2020-11-20T13:56:37.812 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.808+0000 7f0a80996700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0a7c144bd0 0x7f0a7c0b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:37.813 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.808+0000 7f0a81998700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0a7c0b45e0 0x7f0a7c0b1450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45640/0 (socket says 172.21.15.40:45640) 2020-11-20T13:56:37.814 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.808+0000 7f0a81998700 1 -- 172.21.15.40:0/3488391419 learned_addr learned my addr 172.21.15.40:0/3488391419 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:37.814 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.809+0000 7f0a81197700 1 --2- 172.21.15.40:0/3488391419 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0a7c0b6c80 0x7f0a7c0b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:37.815 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.809+0000 7f0a80996700 1 -- 172.21.15.40:0/3488391419 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0a7c0b6c80 msgr2=0x7f0a7c0b31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:37.815 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.809+0000 7f0a80996700 1 --2- 172.21.15.40:0/3488391419 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0a7c0b6c80 0x7f0a7c0b31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.815 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.809+0000 7f0a80996700 1 -- 172.21.15.40:0/3488391419 >> v1:172.21.15.40:6789/0 conn(0x7f0a7c0b45e0 legacy=0x7f0a7c0b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:37.815 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.809+0000 7f0a6ffff700 1 -- 172.21.15.40:0/3488391419 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1331560263 0 0) 0x7f0a7c0b0d10 con 0x7f0a7c0b45e0 2020-11-20T13:56:37.815 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.809+0000 7f0a80996700 1 -- 172.21.15.40:0/3488391419 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a7c0b7060 con 0x7f0a7c144bd0 2020-11-20T13:56:37.816 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.809+0000 7f0a80996700 1 --2- 172.21.15.40:0/3488391419 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a7c144bd0 0x7f0a7c0b40f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f0a78002f30 tx=0x7f0a78006dc0).ready entity=mon.0 client_cookie=6def9fad65fba9aa server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:37.816 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.809+0000 7f0a6ffff700 1 -- 172.21.15.40:0/3488391419 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a780097f0 con 0x7f0a7c144bd0 2020-11-20T13:56:37.816 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.809+0000 7f0a6ffff700 1 -- 172.21.15.40:0/3488391419 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a78009950 con 0x7f0a7c144bd0 2020-11-20T13:56:37.816 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.809+0000 7f0a6ffff700 1 -- 172.21.15.40:0/3488391419 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a78009cb0 con 0x7f0a7c144bd0 2020-11-20T13:56:37.816 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.809+0000 7f0a833fb700 1 -- 172.21.15.40:0/3488391419 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a7c144bd0 msgr2=0x7f0a7c0b40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:37.817 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.809+0000 7f0a833fb700 1 --2- 172.21.15.40:0/3488391419 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a7c144bd0 0x7f0a7c0b40f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f0a78002f30 tx=0x7f0a78006dc0).stop 2020-11-20T13:56:37.817 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.809+0000 7f0a833fb700 1 -- 172.21.15.40:0/3488391419 shutdown_connections 2020-11-20T13:56:37.817 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.809+0000 7f0a833fb700 1 --2- 172.21.15.40:0/3488391419 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0a7c0b6c80 0x7f0a7c0b31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.817 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.809+0000 7f0a833fb700 1 --2- 172.21.15.40:0/3488391419 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a7c144bd0 0x7f0a7c0b40f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.817 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.809+0000 7f0a833fb700 1 -- 172.21.15.40:0/3488391419 >> 172.21.15.40:0/3488391419 conn(0x7f0a7c0ba1d0 msgr2=0x7f0a7c0b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:37.818 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.810+0000 7f0a833fb700 1 -- 172.21.15.40:0/3488391419 shutdown_connections 2020-11-20T13:56:37.818 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.810+0000 7f0a833fb700 1 -- 172.21.15.40:0/3488391419 wait complete. 2020-11-20T13:56:37.818 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.811+0000 7f0a833fb700 1 Processor -- start 2020-11-20T13:56:37.818 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.811+0000 7f0a833fb700 1 -- start start 2020-11-20T13:56:37.821 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.811+0000 7f0a833fb700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a7c0b45e0 0x7f0a7c122640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:37.821 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.811+0000 7f0a833fb700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0a7c0b6c80 0x7f0a7c145e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:37.821 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.811+0000 7f0a833fb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a7c144bd0 0x7f0a7c146300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:37.822 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.811+0000 7f0a833fb700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c147bd0 con 0x7f0a7c144bd0 2020-11-20T13:56:37.822 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.811+0000 7f0a833fb700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c148190 con 0x7f0a7c0b45e0 2020-11-20T13:56:37.823 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.811+0000 7f0a833fb700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c148430 con 0x7f0a7c0b6c80 2020-11-20T13:56:37.823 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.811+0000 7f0a81998700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a7c144bd0 0x7f0a7c146300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:37.823 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.811+0000 7f0a81998700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a7c144bd0 0x7f0a7c146300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55052/0 (socket says 172.21.15.40:55052) 2020-11-20T13:56:37.823 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.811+0000 7f0a81998700 1 -- 172.21.15.40:0/2160795898 learned_addr learned my addr 172.21.15.40:0/2160795898 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:37.823 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.811+0000 7f0a81197700 1 --2- 172.21.15.40:0/2160795898 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a7c0b45e0 0x7f0a7c122640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:37.824 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.811+0000 7f0a80996700 1 --2- 172.21.15.40:0/2160795898 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0a7c0b6c80 0x7f0a7c145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:37.824 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.812+0000 7f0a81998700 1 -- 172.21.15.40:0/2160795898 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0a7c0b6c80 msgr2=0x7f0a7c145e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:37.824 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.812+0000 7f0a81998700 1 --2- 172.21.15.40:0/2160795898 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0a7c0b6c80 0x7f0a7c145e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.824 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.812+0000 7f0a81998700 1 -- 172.21.15.40:0/2160795898 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a7c0b45e0 msgr2=0x7f0a7c122640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:37.824 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.812+0000 7f0a81998700 1 --2- 172.21.15.40:0/2160795898 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a7c0b45e0 0x7f0a7c122640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.825 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.812+0000 7f0a81998700 1 -- 172.21.15.40:0/2160795898 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a7c122b30 con 0x7f0a7c144bd0 2020-11-20T13:56:37.825 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.812+0000 7f0a81998700 1 --2- 172.21.15.40:0/2160795898 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a7c144bd0 0x7f0a7c146300 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f0a74000f00 tx=0x7f0a74007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:37.825 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.812+0000 7f0a6e7fc700 1 -- 172.21.15.40:0/2160795898 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a74005960 con 0x7f0a7c144bd0 2020-11-20T13:56:37.825 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.812+0000 7f0a6e7fc700 1 -- 172.21.15.40:0/2160795898 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a74005ac0 con 0x7f0a7c144bd0 2020-11-20T13:56:37.825 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.812+0000 7f0a833fb700 1 -- 172.21.15.40:0/2160795898 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a7c0ae210 con 0x7f0a7c144bd0 2020-11-20T13:56:37.826 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.812+0000 7f0a6e7fc700 1 -- 172.21.15.40:0/2160795898 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a74005e20 con 0x7f0a7c144bd0 2020-11-20T13:56:37.826 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.812+0000 7f0a833fb700 1 -- 172.21.15.40:0/2160795898 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a7c143f70 con 0x7f0a7c144bd0 2020-11-20T13:56:37.826 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.813+0000 7f0a833fb700 1 -- 172.21.15.40:0/2160795898 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a7c0ae400 con 0x7f0a7c144bd0 2020-11-20T13:56:37.826 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.813+0000 7f0a6e7fc700 1 -- 172.21.15.40:0/2160795898 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f0a74010030 con 0x7f0a7c144bd0 2020-11-20T13:56:37.826 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.813+0000 7f0a6e7fc700 1 --2- 172.21.15.40:0/2160795898 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f0a5c051320 0x7f0a5c0537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:37.827 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.813+0000 7f0a6e7fc700 1 -- 172.21.15.40:0/2160795898 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f0a74068ef0 con 0x7f0a7c144bd0 2020-11-20T13:56:37.827 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.814+0000 7f0a81197700 1 --2- 172.21.15.40:0/2160795898 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f0a5c051320 0x7f0a5c0537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:37.827 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.814+0000 7f0a81197700 1 --2- 172.21.15.40:0/2160795898 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f0a5c051320 0x7f0a5c0537e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f0a7000c000 tx=0x7f0a70008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:37.827 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.817+0000 7f0a6e7fc700 1 -- 172.21.15.40:0/2160795898 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f0a74041110 con 0x7f0a7c144bd0 2020-11-20T13:56:37.977 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.977+0000 7f0a833fb700 1 -- 172.21.15.40:0/2160795898 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0a7c1485f0 con 0x7f0a7c144bd0 2020-11-20T13:56:37.977 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.977+0000 7f0a6e7fc700 1 -- 172.21.15.40:0/2160795898 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+10 (secure 0 0 0) 0x7f0a74044730 con 0x7f0a7c144bd0 2020-11-20T13:56:37.978 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2020-11-20T13:56:37.979 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.978+0000 7f0a833fb700 1 -- 172.21.15.40:0/2160795898 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f0a5c051320 msgr2=0x7f0a5c0537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:37.979 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.978+0000 7f0a833fb700 1 --2- 172.21.15.40:0/2160795898 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f0a5c051320 0x7f0a5c0537e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f0a7000c000 tx=0x7f0a70008040).stop 2020-11-20T13:56:37.979 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.978+0000 7f0a833fb700 1 -- 172.21.15.40:0/2160795898 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a7c144bd0 msgr2=0x7f0a7c146300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:37.980 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.978+0000 7f0a833fb700 1 --2- 172.21.15.40:0/2160795898 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a7c144bd0 0x7f0a7c146300 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f0a74000f00 tx=0x7f0a74007c50).stop 2020-11-20T13:56:37.980 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.979+0000 7f0a833fb700 1 -- 172.21.15.40:0/2160795898 shutdown_connections 2020-11-20T13:56:37.980 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.979+0000 7f0a833fb700 1 --2- 172.21.15.40:0/2160795898 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f0a5c051320 0x7f0a5c0537e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.980 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.979+0000 7f0a833fb700 1 --2- 172.21.15.40:0/2160795898 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a7c144bd0 0x7f0a7c146300 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.980 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.979+0000 7f0a833fb700 1 --2- 172.21.15.40:0/2160795898 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a7c0b45e0 0x7f0a7c122640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.981 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.979+0000 7f0a833fb700 1 --2- 172.21.15.40:0/2160795898 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0a7c0b6c80 0x7f0a7c145e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.981 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.979+0000 7f0a833fb700 1 -- 172.21.15.40:0/2160795898 >> 172.21.15.40:0/2160795898 conn(0x7f0a7c0ba1d0 msgr2=0x7f0a7c0b8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:37.981 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.979+0000 7f0a833fb700 1 -- 172.21.15.40:0/2160795898 shutdown_connections 2020-11-20T13:56:37.981 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:37.979+0000 7f0a833fb700 1 -- 172.21.15.40:0/2160795898 wait complete. 2020-11-20T13:56:37.989 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-20T13:56:38.105 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.103+0000 7fa10f10f700 1 Processor -- start 2020-11-20T13:56:38.105 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.103+0000 7fa10f10f700 1 -- start start 2020-11-20T13:56:38.113 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.104+0000 7fa10f10f700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa10809af20 0x7fa1080b4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:38.113 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.104+0000 7fa10f10f700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa1080b81c0 0x7fa1080b4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:38.114 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.104+0000 7fa10f10f700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa10809bff0 con 0x7fa10809bc10 2020-11-20T13:56:38.114 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.104+0000 7fa10f10f700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1080b8740 con 0x7fa1080b81c0 2020-11-20T13:56:38.114 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.104+0000 7fa10f10f700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1080b89e0 con 0x7fa10809af20 2020-11-20T13:56:38.114 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.104+0000 7fa10ceab700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa10809bc10 0x7fa1080aff40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45652/0 (socket says 172.21.15.40:45652) 2020-11-20T13:56:38.114 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.104+0000 7fa10ceab700 1 -- 172.21.15.40:0/4161290469 learned_addr learned my addr 172.21.15.40:0/4161290469 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:38.115 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.104+0000 7fa107fff700 1 --2- 172.21.15.40:0/4161290469 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa10809af20 0x7fa1080b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:38.115 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.104+0000 7fa10d6ac700 1 --2- 172.21.15.40:0/4161290469 >> v2:172.21.15.116:3300/0 conn(0x7fa1080b81c0 0x7fa1080b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:38.115 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.105+0000 7fa1077fe700 1 -- 172.21.15.40:0/4161290469 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3098400652 0 0) 0x7fa10809bff0 con 0x7fa10809bc10 2020-11-20T13:56:38.115 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.105+0000 7fa1077fe700 1 -- 172.21.15.40:0/4161290469 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0f4003230 con 0x7fa10809bc10 2020-11-20T13:56:38.116 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.105+0000 7fa1077fe700 1 -- 172.21.15.40:0/4161290469 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1674173624 0 0) 0x7fa0f4003230 con 0x7fa10809bc10 2020-11-20T13:56:38.116 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.105+0000 7fa107fff700 1 -- 172.21.15.40:0/4161290469 >> v2:172.21.15.116:3300/0 conn(0x7fa1080b81c0 msgr2=0x7fa1080b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:38.116 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.105+0000 7fa107fff700 1 --2- 172.21.15.40:0/4161290469 >> v2:172.21.15.116:3300/0 conn(0x7fa1080b81c0 0x7fa1080b4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.116 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.105+0000 7fa107fff700 1 -- 172.21.15.40:0/4161290469 >> v1:172.21.15.40:6789/0 conn(0x7fa10809bc10 legacy=0x7fa1080aff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:38.117 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.105+0000 7fa107fff700 1 -- 172.21.15.40:0/4161290469 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1080b8c80 con 0x7fa10809af20 2020-11-20T13:56:38.117 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.105+0000 7fa107fff700 1 --2- 172.21.15.40:0/4161290469 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa10809af20 0x7fa1080b4250 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fa0f8001870 tx=0x7fa0fc006dc0).ready entity=mon.2 client_cookie=73c2d44b28f57daa server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:38.117 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.105+0000 7fa1077fe700 1 -- 172.21.15.40:0/4161290469 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0f80016d0 con 0x7fa10809af20 2020-11-20T13:56:38.118 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.105+0000 7fa1077fe700 1 -- 172.21.15.40:0/4161290469 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0fc00d570 con 0x7fa10809af20 2020-11-20T13:56:38.118 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.105+0000 7fa1077fe700 1 -- 172.21.15.40:0/4161290469 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0fc00d6e0 con 0x7fa10809af20 2020-11-20T13:56:38.118 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.105+0000 7fa10f10f700 1 -- 172.21.15.40:0/4161290469 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa10809af20 msgr2=0x7fa1080b4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:38.119 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.105+0000 7fa10f10f700 1 --2- 172.21.15.40:0/4161290469 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa10809af20 0x7fa1080b4250 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fa0f8001870 tx=0x7fa0fc006dc0).stop 2020-11-20T13:56:38.119 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.106+0000 7fa10f10f700 1 -- 172.21.15.40:0/4161290469 shutdown_connections 2020-11-20T13:56:38.119 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.106+0000 7fa10f10f700 1 --2- 172.21.15.40:0/4161290469 >> v2:172.21.15.116:3300/0 conn(0x7fa1080b81c0 0x7fa1080b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.119 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.106+0000 7fa10f10f700 1 --2- 172.21.15.40:0/4161290469 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa10809af20 0x7fa1080b4250 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.120 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.106+0000 7fa10f10f700 1 -- 172.21.15.40:0/4161290469 >> 172.21.15.40:0/4161290469 conn(0x7fa1080ba380 msgr2=0x7fa1080b6350 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:38.120 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.106+0000 7fa10f10f700 1 -- 172.21.15.40:0/4161290469 shutdown_connections 2020-11-20T13:56:38.120 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.106+0000 7fa10f10f700 1 -- 172.21.15.40:0/4161290469 wait complete. 2020-11-20T13:56:38.120 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.107+0000 7fa10f10f700 1 Processor -- start 2020-11-20T13:56:38.121 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.107+0000 7fa10f10f700 1 -- start start 2020-11-20T13:56:38.121 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.107+0000 7fa10f10f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa10809af20 0x7fa108145110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:38.121 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.107+0000 7fa10f10f700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa10809bc10 0x7fa108099ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:38.121 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.107+0000 7fa10f10f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1080b81c0 0x7fa10809a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:38.121 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.108+0000 7fa10f10f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa108144490 con 0x7fa1080b81c0 2020-11-20T13:56:38.122 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.108+0000 7fa10f10f700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1081456c0 con 0x7fa10809af20 2020-11-20T13:56:38.122 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.108+0000 7fa10f10f700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa108145fc0 con 0x7fa10809bc10 2020-11-20T13:56:38.122 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.108+0000 7fa10d6ac700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1080b81c0 0x7fa10809a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:38.122 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.108+0000 7fa10d6ac700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1080b81c0 0x7fa10809a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55066/0 (socket says 172.21.15.40:55066) 2020-11-20T13:56:38.123 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.108+0000 7fa10d6ac700 1 -- 172.21.15.40:0/2714864696 learned_addr learned my addr 172.21.15.40:0/2714864696 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:38.123 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.108+0000 7fa10ceab700 1 --2- 172.21.15.40:0/2714864696 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa10809af20 0x7fa108145110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:38.123 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.108+0000 7fa107fff700 1 --2- 172.21.15.40:0/2714864696 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa10809bc10 0x7fa108099ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:38.124 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.108+0000 7fa10d6ac700 1 -- 172.21.15.40:0/2714864696 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa10809bc10 msgr2=0x7fa108099ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:38.124 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.108+0000 7fa10d6ac700 1 --2- 172.21.15.40:0/2714864696 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa10809bc10 0x7fa108099ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.124 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.108+0000 7fa10d6ac700 1 -- 172.21.15.40:0/2714864696 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa10809af20 msgr2=0x7fa108145110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:38.125 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.108+0000 7fa10d6ac700 1 --2- 172.21.15.40:0/2714864696 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa10809af20 0x7fa108145110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.125 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.108+0000 7fa10d6ac700 1 -- 172.21.15.40:0/2714864696 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa108146260 con 0x7fa1080b81c0 2020-11-20T13:56:38.125 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.108+0000 7fa10d6ac700 1 --2- 172.21.15.40:0/2714864696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1080b81c0 0x7fa10809a3b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fa100006dd0 tx=0x7fa1000091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:38.126 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.108+0000 7fa105ffb700 1 -- 172.21.15.40:0/2714864696 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa10000c950 con 0x7fa1080b81c0 2020-11-20T13:56:38.126 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.108+0000 7fa105ffb700 1 -- 172.21.15.40:0/2714864696 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa10000cab0 con 0x7fa1080b81c0 2020-11-20T13:56:38.126 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.108+0000 7fa10f10f700 1 -- 172.21.15.40:0/2714864696 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa10813de00 con 0x7fa1080b81c0 2020-11-20T13:56:38.127 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.108+0000 7fa105ffb700 1 -- 172.21.15.40:0/2714864696 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa10000ce10 con 0x7fa1080b81c0 2020-11-20T13:56:38.127 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.108+0000 7fa10f10f700 1 -- 172.21.15.40:0/2714864696 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa10813e270 con 0x7fa1080b81c0 2020-11-20T13:56:38.128 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.109+0000 7fa10f10f700 1 -- 172.21.15.40:0/2714864696 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1081463f0 con 0x7fa1080b81c0 2020-11-20T13:56:38.128 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.109+0000 7fa105ffb700 1 -- 172.21.15.40:0/2714864696 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fa100022070 con 0x7fa1080b81c0 2020-11-20T13:56:38.128 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.110+0000 7fa105ffb700 1 --2- 172.21.15.40:0/2714864696 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa0e8051210 0x7fa0e80536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:38.129 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.110+0000 7fa105ffb700 1 -- 172.21.15.40:0/2714864696 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fa100069770 con 0x7fa1080b81c0 2020-11-20T13:56:38.129 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.110+0000 7fa10ceab700 1 --2- 172.21.15.40:0/2714864696 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa0e8051210 0x7fa0e80536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:38.130 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.110+0000 7fa10ceab700 1 --2- 172.21.15.40:0/2714864696 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa0e8051210 0x7fa0e80536d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fa0f800d000 tx=0x7fa0f8001e50).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:38.131 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.113+0000 7fa105ffb700 1 -- 172.21.15.40:0/2714864696 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fa100041990 con 0x7fa1080b81c0 2020-11-20T13:56:38.328 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.327+0000 7fa10f10f700 1 -- 172.21.15.40:0/2714864696 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa108057050 con 0x7fa1080b81c0 2020-11-20T13:56:38.329 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.328+0000 7fa105ffb700 1 -- 172.21.15.40:0/2714864696 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+2134 (secure 0 0 0) 0x7fa100016070 con 0x7fa1080b81c0 2020-11-20T13:56:38.329 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:56:38.330 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/2019740031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2019740031},{"type":"v1","addr":"172.21.15.116:6835","nonce":2019740031}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T13:56:25.428178+0000","modified":"2020-11-20T13:56:27.432735+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6833/584131564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":584131564},{"type":"v1","addr":"172.21.15.40:6833","nonce":584131564}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-20T13:56:38.331 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.329+0000 7fa10f10f700 1 -- 172.21.15.40:0/2714864696 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa0e8051210 msgr2=0x7fa0e80536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:38.331 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.329+0000 7fa10f10f700 1 --2- 172.21.15.40:0/2714864696 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa0e8051210 0x7fa0e80536d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fa0f800d000 tx=0x7fa0f8001e50).stop 2020-11-20T13:56:38.331 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.329+0000 7fa10f10f700 1 -- 172.21.15.40:0/2714864696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1080b81c0 msgr2=0x7fa10809a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:38.331 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.329+0000 7fa10f10f700 1 --2- 172.21.15.40:0/2714864696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1080b81c0 0x7fa10809a3b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fa100006dd0 tx=0x7fa1000091a0).stop 2020-11-20T13:56:38.332 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.329+0000 7fa10f10f700 1 -- 172.21.15.40:0/2714864696 shutdown_connections 2020-11-20T13:56:38.332 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.329+0000 7fa10f10f700 1 --2- 172.21.15.40:0/2714864696 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa0e8051210 0x7fa0e80536d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.332 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.329+0000 7fa10f10f700 1 --2- 172.21.15.40:0/2714864696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1080b81c0 0x7fa10809a3b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.332 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.329+0000 7fa10f10f700 1 --2- 172.21.15.40:0/2714864696 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa10809af20 0x7fa108145110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.332 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.329+0000 7fa10f10f700 1 --2- 172.21.15.40:0/2714864696 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa10809bc10 0x7fa108099ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.333 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.329+0000 7fa10f10f700 1 -- 172.21.15.40:0/2714864696 >> 172.21.15.40:0/2714864696 conn(0x7fa1080ba380 msgr2=0x7fa1080afed0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:38.333 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.329+0000 7fa10f10f700 1 -- 172.21.15.40:0/2714864696 shutdown_connections 2020-11-20T13:56:38.333 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.329+0000 7fa10f10f700 1 -- 172.21.15.40:0/2714864696 wait complete. 2020-11-20T13:56:38.334 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 7 2020-11-20T13:56:38.542 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T13:56:25.428178+0000', 'modified': '2020-11-20T13:56:27.432735+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4263}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4263': {'gid': 4263, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.40:6833/584131564', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6832', 'nonce': 584131564}, {'type': 'v1', 'addr': '172.21.15.40:6833', 'nonce': 584131564}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T13:56:38.543 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T13:56:38.544 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T13:56:38.682 INFO:teuthology.orchestra.run.smithi040.stdout:{"cluster_fsid":"da1ed737-5174-430f-a3c9-56f6bbfad13c","whoami":0,"id":4263,"want_state":"up:active","state":"up:active","rank_uptime":13.243868435,"mdsmap_epoch":7,"osdmap_epoch":24,"osdmap_epoch_barrier":24,"uptime":19.558412748999999} 2020-11-20T13:56:38.682 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "da1ed737-5174-430f-a3c9-56f6bbfad13c", "id": 4263, "mdsmap_epoch": 7, "osdmap_epoch": 24, "osdmap_epoch_barrier": 24, "rank_uptime": 13.243868435, "state": "up:active", "uptime": 19.558412749, "want_state": "up:active", "whoami": 0 } 2020-11-20T13:56:38.683 INFO:teuthology.run_tasks:Running task ceph-fuse... 2020-11-20T13:56:38.701 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2020-11-20T13:56:38.701 INFO:tasks.ceph_fuse:config is None 2020-11-20T13:56:38.702 INFO:tasks.ceph_fuse:new config is {'client.0': None, 'client.1': None, 'client.2': None, 'client.3': None} 2020-11-20T13:56:38.702 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2020-11-20T13:56:38.703 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-20T13:56:38.817 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.816+0000 7feefb8a2700 1 Processor -- start 2020-11-20T13:56:38.818 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.816+0000 7feefb8a2700 1 -- start start 2020-11-20T13:56:38.824 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.817+0000 7feefb8a2700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7feef409af20 0x7feef40b4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:38.825 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.817+0000 7feefb8a2700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7feef40b81c0 0x7feef40b4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:38.825 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.817+0000 7feefb8a2700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feef409bff0 con 0x7feef409bc10 2020-11-20T13:56:38.825 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.817+0000 7feefb8a2700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feef40b8740 con 0x7feef409af20 2020-11-20T13:56:38.825 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.817+0000 7feefb8a2700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feef40b89e0 con 0x7feef40b81c0 2020-11-20T13:56:38.826 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.817+0000 7feef8e3d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7feef409af20 0x7feef40b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:38.826 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.817+0000 7feef8e3d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7feef409af20 0x7feef40b4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55070/0 (socket says 172.21.15.40:55070) 2020-11-20T13:56:38.826 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.817+0000 7feef8e3d700 1 -- 172.21.15.40:0/1904342564 learned_addr learned my addr 172.21.15.40:0/1904342564 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:38.827 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.817+0000 7feef9e3f700 1 --2- 172.21.15.40:0/1904342564 >> v2:172.21.15.116:3300/0 conn(0x7feef40b81c0 0x7feef40b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:38.827 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.817+0000 7feef8e3d700 1 -- 172.21.15.40:0/1904342564 >> v2:172.21.15.116:3300/0 conn(0x7feef40b81c0 msgr2=0x7feef40b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:38.827 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.817+0000 7feef8e3d700 1 --2- 172.21.15.40:0/1904342564 >> v2:172.21.15.116:3300/0 conn(0x7feef40b81c0 0x7feef40b4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.827 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.818+0000 7feef8e3d700 1 -- 172.21.15.40:0/1904342564 >> v1:172.21.15.116:6789/0 conn(0x7feef409bc10 legacy=0x7feef40aff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:38.828 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.818+0000 7feef8e3d700 1 -- 172.21.15.40:0/1904342564 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef40b8c80 con 0x7feef409af20 2020-11-20T13:56:38.828 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.818+0000 7feef8e3d700 1 --2- 172.21.15.40:0/1904342564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feef409af20 0x7feef40b4250 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7feef40afe50 tx=0x7feef0006e80).ready entity=mon.0 client_cookie=b7daf24a2f49b7d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:38.828 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.818+0000 7feee3fff700 1 -- 172.21.15.40:0/1904342564 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feef00097b0 con 0x7feef409af20 2020-11-20T13:56:38.828 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.818+0000 7feee3fff700 1 -- 172.21.15.40:0/1904342564 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef0009910 con 0x7feef409af20 2020-11-20T13:56:38.829 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.818+0000 7feee3fff700 1 -- 172.21.15.40:0/1904342564 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feef0009a80 con 0x7feef409af20 2020-11-20T13:56:38.829 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.818+0000 7feefb8a2700 1 -- 172.21.15.40:0/1904342564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feef409af20 msgr2=0x7feef40b4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:38.829 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.818+0000 7feefb8a2700 1 --2- 172.21.15.40:0/1904342564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feef409af20 0x7feef40b4250 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7feef40afe50 tx=0x7feef0006e80).stop 2020-11-20T13:56:38.829 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.818+0000 7feefb8a2700 1 -- 172.21.15.40:0/1904342564 shutdown_connections 2020-11-20T13:56:38.830 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.818+0000 7feefb8a2700 1 --2- 172.21.15.40:0/1904342564 >> v2:172.21.15.116:3300/0 conn(0x7feef40b81c0 0x7feef40b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.830 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.818+0000 7feefb8a2700 1 --2- 172.21.15.40:0/1904342564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feef409af20 0x7feef40b4250 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.830 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.818+0000 7feefb8a2700 1 -- 172.21.15.40:0/1904342564 >> 172.21.15.40:0/1904342564 conn(0x7feef40ba380 msgr2=0x7feef40b6350 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:38.830 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.818+0000 7feefb8a2700 1 -- 172.21.15.40:0/1904342564 shutdown_connections 2020-11-20T13:56:38.831 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.818+0000 7feefb8a2700 1 -- 172.21.15.40:0/1904342564 wait complete. 2020-11-20T13:56:38.831 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.819+0000 7feefb8a2700 1 Processor -- start 2020-11-20T13:56:38.831 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.819+0000 7feefb8a2700 1 -- start start 2020-11-20T13:56:38.831 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.820+0000 7feefb8a2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feef409af20 0x7feef40b2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:38.832 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.820+0000 7feefb8a2700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7feef409bc10 0x7feef41399a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:38.832 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.820+0000 7feefb8a2700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feef40b81c0 0x7feef4139e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:38.832 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.820+0000 7feefb8a2700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef4144040 con 0x7feef409af20 2020-11-20T13:56:38.832 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.820+0000 7feefb8a2700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef4145a40 con 0x7feef40b81c0 2020-11-20T13:56:38.832 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.820+0000 7feefb8a2700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feef4145ce0 con 0x7feef409bc10 2020-11-20T13:56:38.833 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.820+0000 7feef963e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feef409af20 0x7feef40b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:38.833 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.820+0000 7feef963e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feef409af20 0x7feef40b2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55080/0 (socket says 172.21.15.40:55080) 2020-11-20T13:56:38.833 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.820+0000 7feef963e700 1 -- 172.21.15.40:0/2902750278 learned_addr learned my addr 172.21.15.40:0/2902750278 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:38.833 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.820+0000 7feef9e3f700 1 --2- 172.21.15.40:0/2902750278 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feef40b81c0 0x7feef4139e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:38.834 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.820+0000 7feef8e3d700 1 --2- 172.21.15.40:0/2902750278 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7feef409bc10 0x7feef41399a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:38.834 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.820+0000 7feef963e700 1 -- 172.21.15.40:0/2902750278 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7feef409bc10 msgr2=0x7feef41399a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:38.834 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.820+0000 7feef963e700 1 --2- 172.21.15.40:0/2902750278 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7feef409bc10 0x7feef41399a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.834 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.820+0000 7feef963e700 1 -- 172.21.15.40:0/2902750278 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feef40b81c0 msgr2=0x7feef4139e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:38.834 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.820+0000 7feef963e700 1 --2- 172.21.15.40:0/2902750278 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feef40b81c0 0x7feef4139e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.835 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.820+0000 7feef963e700 1 -- 172.21.15.40:0/2902750278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef4145f80 con 0x7feef409af20 2020-11-20T13:56:38.835 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.821+0000 7feef963e700 1 --2- 172.21.15.40:0/2902750278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feef409af20 0x7feef40b2660 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7feee4001f50 tx=0x7feee4007b70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:38.835 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.821+0000 7feee27fc700 1 -- 172.21.15.40:0/2902750278 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feee400e4a0 con 0x7feef409af20 2020-11-20T13:56:38.835 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.821+0000 7feee27fc700 1 -- 172.21.15.40:0/2902750278 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee400e600 con 0x7feef409af20 2020-11-20T13:56:38.836 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.821+0000 7feefb8a2700 1 -- 172.21.15.40:0/2902750278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feef4146420 con 0x7feef409af20 2020-11-20T13:56:38.836 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.821+0000 7feee27fc700 1 -- 172.21.15.40:0/2902750278 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feee400e770 con 0x7feef409af20 2020-11-20T13:56:38.836 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.821+0000 7feefb8a2700 1 -- 172.21.15.40:0/2902750278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feef4143b90 con 0x7feef409af20 2020-11-20T13:56:38.836 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.822+0000 7feee27fc700 1 -- 172.21.15.40:0/2902750278 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7feee4010030 con 0x7feef409af20 2020-11-20T13:56:38.836 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.822+0000 7feefb8a2700 1 -- 172.21.15.40:0/2902750278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feef41465b0 con 0x7feef409af20 2020-11-20T13:56:38.837 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.822+0000 7feee27fc700 1 --2- 172.21.15.40:0/2902750278 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7feed4051210 0x7feed40536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:38.837 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.822+0000 7feee27fc700 1 -- 172.21.15.40:0/2902750278 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7feee4068d90 con 0x7feef409af20 2020-11-20T13:56:38.837 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.822+0000 7feef8e3d700 1 --2- 172.21.15.40:0/2902750278 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7feed4051210 0x7feed40536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:38.837 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.823+0000 7feef8e3d700 1 --2- 172.21.15.40:0/2902750278 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7feed4051210 0x7feed40536d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7feef001b000 tx=0x7feef0016000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:38.838 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.826+0000 7feee27fc700 1 -- 172.21.15.40:0/2902750278 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7feee4040fb0 con 0x7feef409af20 2020-11-20T13:56:38.988 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.987+0000 7feefb8a2700 1 -- 172.21.15.40:0/2902750278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feef4057050 con 0x7feef409af20 2020-11-20T13:56:38.989 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.987+0000 7feee27fc700 1 -- 172.21.15.40:0/2902750278 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16347 (secure 0 0 0) 0x7feee40445d0 con 0x7feef409af20 2020-11-20T13:56:38.989 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:56:38.989 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":24,"fsid":"da1ed737-5174-430f-a3c9-56f6bbfad13c","created":"2020-11-20T13:56:04.990651+0000","modified":"2020-11-20T13:56:25.428032+0000","last_up_change":"2020-11-20T13:56:13.893026+0000","last_in_change":"2020-11-20T13:56:13.893026+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T13:56:08.330017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T13:56:17.939684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T13:56:20.563740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T13:56:23.062623+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9ea4c667-8938-46d6-a5ca-092d55c1f79e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":22825},{"type":"v1","addr":"172.21.15.40:6818","nonce":22825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":22825},{"type":"v1","addr":"172.21.15.40:6822","nonce":22825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":22825},{"type":"v1","addr":"172.21.15.40:6830","nonce":22825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":22825},{"type":"v1","addr":"172.21.15.40:6826","nonce":22825}]},"public_addr":"172.21.15.40:6818/22825","cluster_addr":"172.21.15.40:6822/22825","heartbeat_back_addr":"172.21.15.40:6830/22825","heartbeat_front_addr":"172.21.15.40:6826/22825","state":["exists","up"]},{"osd":1,"uuid":"515708d9-5087-4e55-8d14-cdea1c00eacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":22823},{"type":"v1","addr":"172.21.15.40:6801","nonce":22823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":22823},{"type":"v1","addr":"172.21.15.40:6803","nonce":22823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":22823},{"type":"v1","addr":"172.21.15.40:6807","nonce":22823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":22823},{"type":"v1","addr":"172.21.15.40:6805","nonce":22823}]},"public_addr":"172.21.15.40:6801/22823","cluster_addr":"172.21.15.40:6803/22823","heartbeat_back_addr":"172.21.15.40:6807/22823","heartbeat_front_addr":"172.21.15.40:6805/22823","state":["exists","up"]},{"osd":2,"uuid":"4d86bca7-743e-4983-b35f-36676c405b36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":22824},{"type":"v1","addr":"172.21.15.40:6809","nonce":22824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":22824},{"type":"v1","addr":"172.21.15.40:6811","nonce":22824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":22824},{"type":"v1","addr":"172.21.15.40:6815","nonce":22824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":22824},{"type":"v1","addr":"172.21.15.40:6813","nonce":22824}]},"public_addr":"172.21.15.40:6809/22824","cluster_addr":"172.21.15.40:6811/22824","heartbeat_back_addr":"172.21.15.40:6815/22824","heartbeat_front_addr":"172.21.15.40:6813/22824","state":["exists","up"]},{"osd":3,"uuid":"f4dc726b-6b34-4e8b-8eeb-87060b58002d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":22826},{"type":"v1","addr":"172.21.15.40:6819","nonce":22826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":22826},{"type":"v1","addr":"172.21.15.40:6823","nonce":22826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":22826},{"type":"v1","addr":"172.21.15.40:6831","nonce":22826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":22826},{"type":"v1","addr":"172.21.15.40:6827","nonce":22826}]},"public_addr":"172.21.15.40:6819/22826","cluster_addr":"172.21.15.40:6823/22826","heartbeat_back_addr":"172.21.15.40:6831/22826","heartbeat_front_addr":"172.21.15.40:6827/22826","state":["exists","up"]},{"osd":4,"uuid":"25a6d53b-d1f0-4498-b7d5-ffa38661d403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":23181},{"type":"v1","addr":"172.21.15.116:6819","nonce":23181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":23181},{"type":"v1","addr":"172.21.15.116:6821","nonce":23181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":23181},{"type":"v1","addr":"172.21.15.116:6825","nonce":23181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":23181},{"type":"v1","addr":"172.21.15.116:6823","nonce":23181}]},"public_addr":"172.21.15.116:6819/23181","cluster_addr":"172.21.15.116:6821/23181","heartbeat_back_addr":"172.21.15.116:6825/23181","heartbeat_front_addr":"172.21.15.116:6823/23181","state":["exists","up"]},{"osd":5,"uuid":"7d7a5af8-229b-4351-9a21-a3d509cd4635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":23171},{"type":"v1","addr":"172.21.15.116:6803","nonce":23171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":23171},{"type":"v1","addr":"172.21.15.116:6805","nonce":23171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":23171},{"type":"v1","addr":"172.21.15.116:6809","nonce":23171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":23171},{"type":"v1","addr":"172.21.15.116:6807","nonce":23171}]},"public_addr":"172.21.15.116:6803/23171","cluster_addr":"172.21.15.116:6805/23171","heartbeat_back_addr":"172.21.15.116:6809/23171","heartbeat_front_addr":"172.21.15.116:6807/23171","state":["exists","up"]},{"osd":6,"uuid":"8247d248-a140-4e36-9796-9847a97df58d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":23172},{"type":"v1","addr":"172.21.15.116:6811","nonce":23172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":23172},{"type":"v1","addr":"172.21.15.116:6813","nonce":23172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":23172},{"type":"v1","addr":"172.21.15.116:6817","nonce":23172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":23172},{"type":"v1","addr":"172.21.15.116:6815","nonce":23172}]},"public_addr":"172.21.15.116:6811/23172","cluster_addr":"172.21.15.116:6813/23172","heartbeat_back_addr":"172.21.15.116:6817/23172","heartbeat_front_addr":"172.21.15.116:6815/23172","state":["exists","up"]},{"osd":7,"uuid":"472803c4-70b8-499d-b18a-71aaeec6aab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":23196},{"type":"v1","addr":"172.21.15.116:6827","nonce":23196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":23196},{"type":"v1","addr":"172.21.15.116:6829","nonce":23196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":23196},{"type":"v1","addr":"172.21.15.116:6833","nonce":23196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":23196},{"type":"v1","addr":"172.21.15.116:6831","nonce":23196}]},"public_addr":"172.21.15.116:6827/23196","cluster_addr":"172.21.15.116:6829/23196","heartbeat_back_addr":"172.21.15.116:6833/23196","heartbeat_front_addr":"172.21.15.116:6831/23196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.029357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:11.989797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.091097+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.014766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.070860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.071864+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.063400+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.109028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-20T13:56:38.990 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.989+0000 7feefb8a2700 1 -- 172.21.15.40:0/2902750278 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7feed4051210 msgr2=0x7feed40536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:38.991 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.989+0000 7feefb8a2700 1 --2- 172.21.15.40:0/2902750278 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7feed4051210 0x7feed40536d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7feef001b000 tx=0x7feef0016000).stop 2020-11-20T13:56:38.991 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.989+0000 7feefb8a2700 1 -- 172.21.15.40:0/2902750278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feef409af20 msgr2=0x7feef40b2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:38.991 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.989+0000 7feefb8a2700 1 --2- 172.21.15.40:0/2902750278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feef409af20 0x7feef40b2660 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7feee4001f50 tx=0x7feee4007b70).stop 2020-11-20T13:56:38.991 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.989+0000 7feefb8a2700 1 -- 172.21.15.40:0/2902750278 shutdown_connections 2020-11-20T13:56:38.992 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.989+0000 7feefb8a2700 1 --2- 172.21.15.40:0/2902750278 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7feed4051210 0x7feed40536d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.992 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.989+0000 7feefb8a2700 1 --2- 172.21.15.40:0/2902750278 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feef40b81c0 0x7feef4139e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.992 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.989+0000 7feefb8a2700 1 --2- 172.21.15.40:0/2902750278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feef409af20 0x7feef40b2660 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.992 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.989+0000 7feefb8a2700 1 --2- 172.21.15.40:0/2902750278 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7feef409bc10 0x7feef41399a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.992 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.989+0000 7feefb8a2700 1 -- 172.21.15.40:0/2902750278 >> 172.21.15.40:0/2902750278 conn(0x7feef40ba380 msgr2=0x7feef40b4520 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:38.993 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.989+0000 7feefb8a2700 1 -- 172.21.15.40:0/2902750278 shutdown_connections 2020-11-20T13:56:38.993 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:38.989+0000 7feefb8a2700 1 -- 172.21.15.40:0/2902750278 wait complete. 2020-11-20T13:56:39.001 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T13:56:08.330017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T13:56:17.939684+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T13:56:20.563740+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T13:56:23.062623+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T13:56:39.002 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-20T13:56:39.119 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.118+0000 7faafd667700 1 Processor -- start 2020-11-20T13:56:39.120 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.118+0000 7faafd667700 1 -- start start 2020-11-20T13:56:39.122 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.119+0000 7faafd667700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7faaf80b3960 0x7faaf805faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:39.122 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.119+0000 7faafd667700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaf80b5fd0 con 0x7faaf8144390 2020-11-20T13:56:39.122 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.119+0000 7faafd667700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaf80b05f0 con 0x7faaf80b5bf0 2020-11-20T13:56:39.123 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.119+0000 7faafd667700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaf8144870 con 0x7faaf80b3960 2020-11-20T13:56:39.123 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.119+0000 7faaf77fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7faaf8144390 0x7faaf814cf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45682/0 (socket says 172.21.15.40:45682) 2020-11-20T13:56:39.123 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.119+0000 7faaf77fe700 1 -- 172.21.15.40:0/3438528151 learned_addr learned my addr 172.21.15.40:0/3438528151 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:39.123 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.119+0000 7faaf67fc700 1 --2- 172.21.15.40:0/3438528151 >> v2:172.21.15.116:3300/0 conn(0x7faaf80b3960 0x7faaf805faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:39.124 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.119+0000 7faaf5ffb700 1 -- 172.21.15.40:0/3438528151 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3490385953 0 0) 0x7faaf80b5fd0 con 0x7faaf8144390 2020-11-20T13:56:39.124 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.119+0000 7faaf5ffb700 1 -- 172.21.15.40:0/3438528151 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faadc003230 con 0x7faaf8144390 2020-11-20T13:56:39.124 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.120+0000 7faaf67fc700 1 -- 172.21.15.40:0/3438528151 >> v1:172.21.15.116:6789/0 conn(0x7faaf80b5bf0 legacy=0x7faaf805f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:39.124 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.120+0000 7faaf67fc700 1 -- 172.21.15.40:0/3438528151 >> v1:172.21.15.40:6789/0 conn(0x7faaf8144390 legacy=0x7faaf814cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:39.124 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.120+0000 7faaf5ffb700 1 -- 172.21.15.40:0/3438528151 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3193679258 0 0) 0x7faaf80b05f0 con 0x7faaf80b5bf0 2020-11-20T13:56:39.125 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.120+0000 7faaf67fc700 1 -- 172.21.15.40:0/3438528151 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf80b0c30 con 0x7faaf80b3960 2020-11-20T13:56:39.125 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.120+0000 7faaf67fc700 1 --2- 172.21.15.40:0/3438528151 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faaf80b3960 0x7faaf805faa0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7faae80032b0 tx=0x7faae800a360).ready entity=mon.1 client_cookie=2ece91edbc636a76 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:39.125 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.120+0000 7faaf5ffb700 1 -- 172.21.15.40:0/3438528151 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faae8010410 con 0x7faaf80b3960 2020-11-20T13:56:39.125 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.120+0000 7faaf5ffb700 1 -- 172.21.15.40:0/3438528151 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faae8010570 con 0x7faaf80b3960 2020-11-20T13:56:39.126 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.120+0000 7faaf5ffb700 1 -- 172.21.15.40:0/3438528151 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faae80108d0 con 0x7faaf80b3960 2020-11-20T13:56:39.126 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.120+0000 7faafd667700 1 -- 172.21.15.40:0/3438528151 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faaf80b3960 msgr2=0x7faaf805faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:39.126 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.120+0000 7faafd667700 1 --2- 172.21.15.40:0/3438528151 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faaf80b3960 0x7faaf805faa0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7faae80032b0 tx=0x7faae800a360).stop 2020-11-20T13:56:39.126 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.120+0000 7faafd667700 1 -- 172.21.15.40:0/3438528151 shutdown_connections 2020-11-20T13:56:39.126 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.120+0000 7faafd667700 1 --2- 172.21.15.40:0/3438528151 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faaf80b3960 0x7faaf805faa0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:39.127 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.120+0000 7faafd667700 1 -- 172.21.15.40:0/3438528151 >> 172.21.15.40:0/3438528151 conn(0x7faaf80ba1d0 msgr2=0x7faaf80b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:39.127 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.121+0000 7faafd667700 1 -- 172.21.15.40:0/3438528151 shutdown_connections 2020-11-20T13:56:39.127 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.121+0000 7faafd667700 1 -- 172.21.15.40:0/3438528151 wait complete. 2020-11-20T13:56:39.130 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.121+0000 7faafd667700 1 Processor -- start 2020-11-20T13:56:39.130 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.122+0000 7faafd667700 1 -- start start 2020-11-20T13:56:39.130 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.122+0000 7faafd667700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7faaf80b3960 0x7faaf81428b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:39.130 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.122+0000 7faafd667700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faaf80b5bf0 0x7faaf8155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:39.131 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.122+0000 7faafd667700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faaf8144390 0x7faaf81407c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:39.131 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.122+0000 7faafd667700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf81560b0 con 0x7faaf80b5bf0 2020-11-20T13:56:39.131 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.122+0000 7faafd667700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf813fde0 con 0x7faaf8144390 2020-11-20T13:56:39.131 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.122+0000 7faafd667700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaf8140080 con 0x7faaf80b3960 2020-11-20T13:56:39.131 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.122+0000 7faaf67fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faaf80b5bf0 0x7faaf8155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:39.132 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.122+0000 7faaf67fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faaf80b5bf0 0x7faaf8155bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55092/0 (socket says 172.21.15.40:55092) 2020-11-20T13:56:39.132 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.122+0000 7faaf67fc700 1 -- 172.21.15.40:0/265879112 learned_addr learned my addr 172.21.15.40:0/265879112 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:39.132 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.122+0000 7faaf77fe700 1 --2- 172.21.15.40:0/265879112 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faaf8144390 0x7faaf81407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:39.132 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.122+0000 7faaf6ffd700 1 --2- 172.21.15.40:0/265879112 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7faaf80b3960 0x7faaf81428b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:39.132 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.123+0000 7faaf67fc700 1 -- 172.21.15.40:0/265879112 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7faaf80b3960 msgr2=0x7faaf81428b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:39.133 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.123+0000 7faaf67fc700 1 --2- 172.21.15.40:0/265879112 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7faaf80b3960 0x7faaf81428b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:39.133 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.123+0000 7faaf67fc700 1 -- 172.21.15.40:0/265879112 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faaf8144390 msgr2=0x7faaf81407c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:39.133 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.123+0000 7faaf67fc700 1 --2- 172.21.15.40:0/265879112 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faaf8144390 0x7faaf81407c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:39.133 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.123+0000 7faaf67fc700 1 -- 172.21.15.40:0/265879112 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf8140320 con 0x7faaf80b5bf0 2020-11-20T13:56:39.134 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.123+0000 7faaf67fc700 1 --2- 172.21.15.40:0/265879112 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faaf80b5bf0 0x7faaf8155bc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7faae80032b0 tx=0x7faaf80b0780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:39.134 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.123+0000 7faae7fff700 1 -- 172.21.15.40:0/265879112 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faae8019870 con 0x7faaf80b5bf0 2020-11-20T13:56:39.134 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.123+0000 7faae7fff700 1 -- 172.21.15.40:0/265879112 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faae80199d0 con 0x7faaf80b5bf0 2020-11-20T13:56:39.134 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.123+0000 7faafd667700 1 -- 172.21.15.40:0/265879112 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaf8140cb0 con 0x7faaf80b5bf0 2020-11-20T13:56:39.134 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.123+0000 7faae7fff700 1 -- 172.21.15.40:0/265879112 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faae8019d30 con 0x7faaf80b5bf0 2020-11-20T13:56:39.135 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.123+0000 7faafd667700 1 -- 172.21.15.40:0/265879112 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaf813e720 con 0x7faaf80b5bf0 2020-11-20T13:56:39.135 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.124+0000 7faafd667700 1 -- 172.21.15.40:0/265879112 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaf81404b0 con 0x7faaf80b5bf0 2020-11-20T13:56:39.135 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.124+0000 7faae7fff700 1 -- 172.21.15.40:0/265879112 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7faae800e040 con 0x7faaf80b5bf0 2020-11-20T13:56:39.135 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.124+0000 7faae7fff700 1 --2- 172.21.15.40:0/265879112 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7faad40513c0 0x7faad4053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:39.135 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.124+0000 7faae7fff700 1 -- 172.21.15.40:0/265879112 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7faae806c220 con 0x7faaf80b5bf0 2020-11-20T13:56:39.136 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.124+0000 7faaf6ffd700 1 --2- 172.21.15.40:0/265879112 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7faad40513c0 0x7faad4053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:39.136 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.125+0000 7faaf6ffd700 1 --2- 172.21.15.40:0/265879112 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7faad40513c0 0x7faad4053880 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7faaec00d000 tx=0x7faaec009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:39.136 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.128+0000 7faae7fff700 1 -- 172.21.15.40:0/265879112 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7faae8044440 con 0x7faaf80b5bf0 2020-11-20T13:56:39.287 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.284+0000 7faafd667700 1 -- 172.21.15.40:0/265879112 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7faaf8057050 con 0x7faaf80b5bf0 2020-11-20T13:56:39.288 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.284+0000 7faae7fff700 1 -- 172.21.15.40:0/265879112 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v24) v1 ==== 110+0+10 (secure 0 0 0) 0x7faae8047a60 con 0x7faaf80b5bf0 2020-11-20T13:56:39.288 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.286+0000 7faafd667700 1 -- 172.21.15.40:0/265879112 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7faad40513c0 msgr2=0x7faad4053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:39.288 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.286+0000 7faafd667700 1 --2- 172.21.15.40:0/265879112 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7faad40513c0 0x7faad4053880 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7faaec00d000 tx=0x7faaec009040).stop 2020-11-20T13:56:39.288 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.286+0000 7faafd667700 1 -- 172.21.15.40:0/265879112 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faaf80b5bf0 msgr2=0x7faaf8155bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:39.289 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.286+0000 7faafd667700 1 --2- 172.21.15.40:0/265879112 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faaf80b5bf0 0x7faaf8155bc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7faae80032b0 tx=0x7faaf80b0780).stop 2020-11-20T13:56:39.289 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.286+0000 7faafd667700 1 -- 172.21.15.40:0/265879112 shutdown_connections 2020-11-20T13:56:39.289 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.286+0000 7faafd667700 1 --2- 172.21.15.40:0/265879112 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7faad40513c0 0x7faad4053880 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:39.289 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.286+0000 7faafd667700 1 --2- 172.21.15.40:0/265879112 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7faaf80b3960 0x7faaf81428b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:39.289 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.286+0000 7faafd667700 1 --2- 172.21.15.40:0/265879112 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faaf8144390 0x7faaf81407c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:39.290 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.286+0000 7faafd667700 1 --2- 172.21.15.40:0/265879112 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faaf80b5bf0 0x7faaf8155bc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:39.290 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.286+0000 7faafd667700 1 -- 172.21.15.40:0/265879112 >> 172.21.15.40:0/265879112 conn(0x7faaf80ba1d0 msgr2=0x7faaf80b7c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:39.290 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.286+0000 7faafd667700 1 -- 172.21.15.40:0/265879112 shutdown_connections 2020-11-20T13:56:39.290 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.286+0000 7faafd667700 1 -- 172.21.15.40:0/265879112 wait complete. 2020-11-20T13:56:39.291 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2020-11-20T13:56:39.297 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-20T13:56:39.414 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.412+0000 7fa8c104f700 1 Processor -- start 2020-11-20T13:56:39.414 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.412+0000 7fa8c104f700 1 -- start start 2020-11-20T13:56:39.416 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.413+0000 7fa8c104f700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa8bc0b6c80 0x7fa8bc0b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:39.416 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.413+0000 7fa8c104f700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa8bc144bd0 0x7fa8bc0b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:39.417 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.413+0000 7fa8c104f700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8bc0b0d10 con 0x7fa8bc0b45e0 2020-11-20T13:56:39.417 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.413+0000 7fa8c104f700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8bc145030 con 0x7fa8bc144bd0 2020-11-20T13:56:39.417 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.413+0000 7fa8c104f700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8bc0b0fb0 con 0x7fa8bc0b6c80 2020-11-20T13:56:39.417 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.414+0000 7fa8ba59c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa8bc144bd0 0x7fa8bc0b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:39.417 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.414+0000 7fa8bb59e700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa8bc0b45e0 0x7fa8bc0b1450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45694/0 (socket says 172.21.15.40:45694) 2020-11-20T13:56:39.418 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.414+0000 7fa8bb59e700 1 -- 172.21.15.40:0/3725860058 learned_addr learned my addr 172.21.15.40:0/3725860058 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:39.418 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.414+0000 7fa8bad9d700 1 --2- 172.21.15.40:0/3725860058 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa8bc0b6c80 0x7fa8bc0b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:39.418 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.414+0000 7fa8ba59c700 1 -- 172.21.15.40:0/3725860058 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa8bc0b6c80 msgr2=0x7fa8bc0b31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:39.419 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.414+0000 7fa8ba59c700 1 --2- 172.21.15.40:0/3725860058 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa8bc0b6c80 0x7fa8bc0b31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:39.419 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.414+0000 7fa8ba59c700 1 -- 172.21.15.40:0/3725860058 >> v1:172.21.15.40:6789/0 conn(0x7fa8bc0b45e0 legacy=0x7fa8bc0b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:39.419 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.414+0000 7fa8ba59c700 1 -- 172.21.15.40:0/3725860058 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8bc0b7060 con 0x7fa8bc144bd0 2020-11-20T13:56:39.419 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.414+0000 7fa8b9d9b700 1 -- 172.21.15.40:0/3725860058 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1011496958 0 0) 0x7fa8bc0b0d10 con 0x7fa8bc0b45e0 2020-11-20T13:56:39.419 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.414+0000 7fa8ba59c700 1 --2- 172.21.15.40:0/3725860058 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8bc144bd0 0x7fa8bc0b40f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fa8ac002f30 tx=0x7fa8ac006dc0).ready entity=mon.0 client_cookie=af975178b070f7a2 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:39.420 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.414+0000 7fa8b9d9b700 1 -- 172.21.15.40:0/3725860058 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8ac009760 con 0x7fa8bc144bd0 2020-11-20T13:56:39.420 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.414+0000 7fa8b9d9b700 1 -- 172.21.15.40:0/3725860058 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8ac0098c0 con 0x7fa8bc144bd0 2020-11-20T13:56:39.420 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.414+0000 7fa8b9d9b700 1 -- 172.21.15.40:0/3725860058 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8ac009c20 con 0x7fa8bc144bd0 2020-11-20T13:56:39.420 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.414+0000 7fa8c104f700 1 -- 172.21.15.40:0/3725860058 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8bc144bd0 msgr2=0x7fa8bc0b40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:39.420 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.414+0000 7fa8c104f700 1 --2- 172.21.15.40:0/3725860058 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8bc144bd0 0x7fa8bc0b40f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fa8ac002f30 tx=0x7fa8ac006dc0).stop 2020-11-20T13:56:39.421 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.415+0000 7fa8c104f700 1 -- 172.21.15.40:0/3725860058 shutdown_connections 2020-11-20T13:56:39.421 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.415+0000 7fa8c104f700 1 --2- 172.21.15.40:0/3725860058 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa8bc0b6c80 0x7fa8bc0b31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:39.421 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.415+0000 7fa8c104f700 1 --2- 172.21.15.40:0/3725860058 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8bc144bd0 0x7fa8bc0b40f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:39.421 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.415+0000 7fa8c104f700 1 -- 172.21.15.40:0/3725860058 >> 172.21.15.40:0/3725860058 conn(0x7fa8bc0ba1d0 msgr2=0x7fa8bc0b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:39.421 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.415+0000 7fa8c104f700 1 -- 172.21.15.40:0/3725860058 shutdown_connections 2020-11-20T13:56:39.422 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.415+0000 7fa8c104f700 1 -- 172.21.15.40:0/3725860058 wait complete. 2020-11-20T13:56:39.425 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.416+0000 7fa8c104f700 1 Processor -- start 2020-11-20T13:56:39.425 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.416+0000 7fa8c104f700 1 -- start start 2020-11-20T13:56:39.425 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.416+0000 7fa8c104f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8bc0b45e0 0x7fa8bc0b7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:39.425 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.416+0000 7fa8c104f700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa8bc0b6c80 0x7fa8bc159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:39.426 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.416+0000 7fa8c104f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8bc144bd0 0x7fa8bc15a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:39.426 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.416+0000 7fa8c104f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8bc131080 con 0x7fa8bc0b45e0 2020-11-20T13:56:39.426 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.416+0000 7fa8c104f700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8bc131320 con 0x7fa8bc144bd0 2020-11-20T13:56:39.426 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.416+0000 7fa8c104f700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8bc1315c0 con 0x7fa8bc0b6c80 2020-11-20T13:56:39.426 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.417+0000 7fa8bad9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8bc0b45e0 0x7fa8bc0b7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:39.427 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.417+0000 7fa8bad9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8bc0b45e0 0x7fa8bc0b7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55104/0 (socket says 172.21.15.40:55104) 2020-11-20T13:56:39.427 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.417+0000 7fa8bad9d700 1 -- 172.21.15.40:0/1194589102 learned_addr learned my addr 172.21.15.40:0/1194589102 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:39.427 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.417+0000 7fa8ba59c700 1 --2- 172.21.15.40:0/1194589102 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa8bc0b6c80 0x7fa8bc159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:39.427 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.417+0000 7fa8bb59e700 1 --2- 172.21.15.40:0/1194589102 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8bc144bd0 0x7fa8bc15a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:39.428 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.417+0000 7fa8bad9d700 1 -- 172.21.15.40:0/1194589102 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa8bc0b6c80 msgr2=0x7fa8bc159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:39.428 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.417+0000 7fa8bad9d700 1 --2- 172.21.15.40:0/1194589102 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa8bc0b6c80 0x7fa8bc159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:39.428 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.417+0000 7fa8bad9d700 1 -- 172.21.15.40:0/1194589102 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8bc144bd0 msgr2=0x7fa8bc15a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:39.428 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.417+0000 7fa8bad9d700 1 --2- 172.21.15.40:0/1194589102 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8bc144bd0 0x7fa8bc15a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:39.428 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.417+0000 7fa8bad9d700 1 -- 172.21.15.40:0/1194589102 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8bc15a640 con 0x7fa8bc0b45e0 2020-11-20T13:56:39.429 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.417+0000 7fa8ba59c700 1 --2- 172.21.15.40:0/1194589102 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa8bc0b6c80 0x7fa8bc159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:39.429 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.417+0000 7fa8bad9d700 1 --2- 172.21.15.40:0/1194589102 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8bc0b45e0 0x7fa8bc0b7b60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7fa8a400c000 tx=0x7fa8a4006d60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:39.429 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.417+0000 7fa8b3fff700 1 -- 172.21.15.40:0/1194589102 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8a400ba80 con 0x7fa8bc0b45e0 2020-11-20T13:56:39.429 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.417+0000 7fa8b3fff700 1 -- 172.21.15.40:0/1194589102 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8a400bbe0 con 0x7fa8bc0b45e0 2020-11-20T13:56:39.429 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.417+0000 7fa8b3fff700 1 -- 172.21.15.40:0/1194589102 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8a40123f0 con 0x7fa8bc0b45e0 2020-11-20T13:56:39.430 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.417+0000 7fa8c104f700 1 -- 172.21.15.40:0/1194589102 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8bc0b8050 con 0x7fa8bc0b45e0 2020-11-20T13:56:39.430 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.417+0000 7fa8c104f700 1 -- 172.21.15.40:0/1194589102 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8bc063760 con 0x7fa8bc0b45e0 2020-11-20T13:56:39.430 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.418+0000 7fa8c104f700 1 -- 172.21.15.40:0/1194589102 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8bc0b8330 con 0x7fa8bc0b45e0 2020-11-20T13:56:39.430 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.418+0000 7fa8b3fff700 1 -- 172.21.15.40:0/1194589102 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fa8a4012550 con 0x7fa8bc0b45e0 2020-11-20T13:56:39.430 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.418+0000 7fa8b3fff700 1 --2- 172.21.15.40:0/1194589102 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa8a0051320 0x7fa8a00537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:39.431 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.419+0000 7fa8b3fff700 1 -- 172.21.15.40:0/1194589102 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fa8a4069590 con 0x7fa8bc0b45e0 2020-11-20T13:56:39.431 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.419+0000 7fa8ba59c700 1 --2- 172.21.15.40:0/1194589102 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa8a0051320 0x7fa8a00537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:39.431 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.419+0000 7fa8ba59c700 1 --2- 172.21.15.40:0/1194589102 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa8a0051320 0x7fa8a00537e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fa8ac018000 tx=0x7fa8ac015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:39.431 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.422+0000 7fa8b3fff700 1 -- 172.21.15.40:0/1194589102 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fa8a4041750 con 0x7fa8bc0b45e0 2020-11-20T13:56:39.584 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.583+0000 7fa8c104f700 1 -- 172.21.15.40:0/1194589102 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa8bc131780 con 0x7fa8bc0b45e0 2020-11-20T13:56:39.585 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.583+0000 7fa8b3fff700 1 -- 172.21.15.40:0/1194589102 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa8a4044d70 con 0x7fa8bc0b45e0 2020-11-20T13:56:39.585 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2020-11-20T13:56:39.586 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.585+0000 7fa8c104f700 1 -- 172.21.15.40:0/1194589102 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa8a0051320 msgr2=0x7fa8a00537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:39.587 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.585+0000 7fa8c104f700 1 --2- 172.21.15.40:0/1194589102 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa8a0051320 0x7fa8a00537e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fa8ac018000 tx=0x7fa8ac015000).stop 2020-11-20T13:56:39.587 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.585+0000 7fa8c104f700 1 -- 172.21.15.40:0/1194589102 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8bc0b45e0 msgr2=0x7fa8bc0b7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:39.587 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.585+0000 7fa8c104f700 1 --2- 172.21.15.40:0/1194589102 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8bc0b45e0 0x7fa8bc0b7b60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7fa8a400c000 tx=0x7fa8a4006d60).stop 2020-11-20T13:56:39.587 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.585+0000 7fa8c104f700 1 -- 172.21.15.40:0/1194589102 shutdown_connections 2020-11-20T13:56:39.588 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.585+0000 7fa8c104f700 1 --2- 172.21.15.40:0/1194589102 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa8a0051320 0x7fa8a00537e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:39.588 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.585+0000 7fa8c104f700 1 --2- 172.21.15.40:0/1194589102 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8bc144bd0 0x7fa8bc15a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:39.588 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.585+0000 7fa8c104f700 1 --2- 172.21.15.40:0/1194589102 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8bc0b45e0 0x7fa8bc0b7b60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:39.588 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.585+0000 7fa8c104f700 1 --2- 172.21.15.40:0/1194589102 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa8bc0b6c80 0x7fa8bc159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:39.588 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.585+0000 7fa8c104f700 1 -- 172.21.15.40:0/1194589102 >> 172.21.15.40:0/1194589102 conn(0x7fa8bc0ba1d0 msgr2=0x7fa8bc144200 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:39.589 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.585+0000 7fa8c104f700 1 -- 172.21.15.40:0/1194589102 shutdown_connections 2020-11-20T13:56:39.590 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.585+0000 7fa8c104f700 1 -- 172.21.15.40:0/1194589102 wait complete. 2020-11-20T13:56:39.597 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-20T13:56:39.716 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.713+0000 7f5fbdb84700 1 Processor -- start 2020-11-20T13:56:39.716 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.713+0000 7f5fbdb84700 1 -- start start 2020-11-20T13:56:39.717 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.714+0000 7f5fbdb84700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5fb80b31f0 0x7f5fb8144bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:39.717 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.714+0000 7f5fbdb84700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5fb80b1450 0x7f5fb80b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:39.717 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.714+0000 7f5fbdb84700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fb80b1dd0 con 0x7f5fb80b6c80 2020-11-20T13:56:39.717 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.714+0000 7f5fbdb84700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fb80b0d10 con 0x7f5fb80b31f0 2020-11-20T13:56:39.717 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.714+0000 7f5fbdb84700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fb80b0fb0 con 0x7f5fb80b1450 2020-11-20T13:56:39.718 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.714+0000 7f5fb6ffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5fb80b31f0 0x7f5fb8144bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:39.718 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.714+0000 7f5fb6ffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5fb80b31f0 0x7f5fb8144bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55114/0 (socket says 172.21.15.40:55114) 2020-11-20T13:56:39.718 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.714+0000 7f5fb7fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5fb80b1450 0x7f5fb80b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:39.718 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.714+0000 7f5fb6ffd700 1 -- 172.21.15.40:0/4122507726 learned_addr learned my addr 172.21.15.40:0/4122507726 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:39.719 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.715+0000 7f5fb6ffd700 1 -- 172.21.15.40:0/4122507726 >> v2:172.21.15.116:3300/0 conn(0x7f5fb80b1450 msgr2=0x7f5fb80b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:39.719 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.715+0000 7f5fb6ffd700 1 --2- 172.21.15.40:0/4122507726 >> v2:172.21.15.116:3300/0 conn(0x7f5fb80b1450 0x7f5fb80b18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:39.719 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.715+0000 7f5fb6ffd700 1 -- 172.21.15.40:0/4122507726 >> v1:172.21.15.116:6789/0 conn(0x7f5fb80b6c80 legacy=0x7f5fb80b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:39.719 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.715+0000 7f5fb6ffd700 1 -- 172.21.15.40:0/4122507726 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb80b7060 con 0x7f5fb80b31f0 2020-11-20T13:56:39.719 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.715+0000 7f5fb6ffd700 1 --2- 172.21.15.40:0/4122507726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fb80b31f0 0x7f5fb8144bd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f5fac002f30 tx=0x7f5fac006e80).ready entity=mon.0 client_cookie=66274a2a22509f2b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:39.720 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.715+0000 7f5fb67fc700 1 -- 172.21.15.40:0/4122507726 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fac0097f0 con 0x7f5fb80b31f0 2020-11-20T13:56:39.720 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.715+0000 7f5fb67fc700 1 -- 172.21.15.40:0/4122507726 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fac009950 con 0x7f5fb80b31f0 2020-11-20T13:56:39.720 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.715+0000 7f5fb67fc700 1 -- 172.21.15.40:0/4122507726 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fac009cb0 con 0x7f5fb80b31f0 2020-11-20T13:56:39.720 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.715+0000 7f5fbdb84700 1 -- 172.21.15.40:0/4122507726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fb80b31f0 msgr2=0x7f5fb8144bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:39.720 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.715+0000 7f5fbdb84700 1 --2- 172.21.15.40:0/4122507726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fb80b31f0 0x7f5fb8144bd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f5fac002f30 tx=0x7f5fac006e80).stop 2020-11-20T13:56:39.723 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.715+0000 7f5fbdb84700 1 -- 172.21.15.40:0/4122507726 shutdown_connections 2020-11-20T13:56:39.723 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.715+0000 7f5fbdb84700 1 --2- 172.21.15.40:0/4122507726 >> v2:172.21.15.116:3300/0 conn(0x7f5fb80b1450 0x7f5fb80b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:39.724 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.715+0000 7f5fbdb84700 1 --2- 172.21.15.40:0/4122507726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fb80b31f0 0x7f5fb8144bd0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:39.724 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.715+0000 7f5fbdb84700 1 -- 172.21.15.40:0/4122507726 >> 172.21.15.40:0/4122507726 conn(0x7f5fb80ba1d0 msgr2=0x7f5fb80b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:39.724 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.715+0000 7f5fbdb84700 1 -- 172.21.15.40:0/4122507726 shutdown_connections 2020-11-20T13:56:39.724 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.715+0000 7f5fbdb84700 1 -- 172.21.15.40:0/4122507726 wait complete. 2020-11-20T13:56:39.724 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.716+0000 7f5fbdb84700 1 Processor -- start 2020-11-20T13:56:39.725 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.717+0000 7f5fbdb84700 1 -- start start 2020-11-20T13:56:39.725 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.717+0000 7f5fbdb84700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5fb80b1450 0x7f5fb80676b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:39.725 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.717+0000 7f5fbdb84700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5fb80b31f0 0x7f5fb812bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:39.725 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.717+0000 7f5fbdb84700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fb80b6c80 0x7f5fb8063f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:39.725 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.717+0000 7f5fbdb84700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb8157680 con 0x7f5fb80b6c80 2020-11-20T13:56:39.726 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.717+0000 7f5fbdb84700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb81573e0 con 0x7f5fb80b31f0 2020-11-20T13:56:39.726 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.717+0000 7f5fbdb84700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb81532e0 con 0x7f5fb80b1450 2020-11-20T13:56:39.726 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.717+0000 7f5fb7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fb80b6c80 0x7f5fb8063f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:39.726 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.717+0000 7f5fb7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fb80b6c80 0x7f5fb8063f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55122/0 (socket says 172.21.15.40:55122) 2020-11-20T13:56:39.726 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.717+0000 7f5fb7fff700 1 -- 172.21.15.40:0/2767951368 learned_addr learned my addr 172.21.15.40:0/2767951368 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:39.727 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.717+0000 7f5fb77fe700 1 --2- 172.21.15.40:0/2767951368 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5fb80b1450 0x7f5fb80676b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:39.727 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.717+0000 7f5fb6ffd700 1 --2- 172.21.15.40:0/2767951368 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5fb80b31f0 0x7f5fb812bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:39.727 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.718+0000 7f5fb7fff700 1 -- 172.21.15.40:0/2767951368 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5fb80b1450 msgr2=0x7f5fb80676b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:39.727 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.718+0000 7f5fb7fff700 1 --2- 172.21.15.40:0/2767951368 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5fb80b1450 0x7f5fb80676b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:39.727 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.718+0000 7f5fb7fff700 1 -- 172.21.15.40:0/2767951368 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5fb80b31f0 msgr2=0x7f5fb812bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:39.728 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.718+0000 7f5fb7fff700 1 --2- 172.21.15.40:0/2767951368 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5fb80b31f0 0x7f5fb812bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:39.728 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.718+0000 7f5fb7fff700 1 -- 172.21.15.40:0/2767951368 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb8157100 con 0x7f5fb80b6c80 2020-11-20T13:56:39.728 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.718+0000 7f5fb6ffd700 1 --2- 172.21.15.40:0/2767951368 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5fb80b31f0 0x7f5fb812bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:39.728 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.718+0000 7f5fb7fff700 1 --2- 172.21.15.40:0/2767951368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fb80b6c80 0x7f5fb8063f00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f5fb000c000 tx=0x7f5fb00091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:39.728 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.718+0000 7f5fb4ff9700 1 -- 172.21.15.40:0/2767951368 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fb000c9a0 con 0x7f5fb80b6c80 2020-11-20T13:56:39.729 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.718+0000 7f5fb4ff9700 1 -- 172.21.15.40:0/2767951368 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb000cb00 con 0x7f5fb80b6c80 2020-11-20T13:56:39.729 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.718+0000 7f5fbdb84700 1 -- 172.21.15.40:0/2767951368 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fb80655d0 con 0x7f5fb80b6c80 2020-11-20T13:56:39.729 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.718+0000 7f5fb4ff9700 1 -- 172.21.15.40:0/2767951368 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fb000ce60 con 0x7f5fb80b6c80 2020-11-20T13:56:39.729 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.718+0000 7f5fbdb84700 1 -- 172.21.15.40:0/2767951368 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fb8064630 con 0x7f5fb80b6c80 2020-11-20T13:56:39.729 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.719+0000 7f5fbdb84700 1 -- 172.21.15.40:0/2767951368 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fb80657c0 con 0x7f5fb80b6c80 2020-11-20T13:56:39.730 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.719+0000 7f5fb4ff9700 1 -- 172.21.15.40:0/2767951368 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f5fb0022070 con 0x7f5fb80b6c80 2020-11-20T13:56:39.730 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.719+0000 7f5fb4ff9700 1 --2- 172.21.15.40:0/2767951368 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f5f98051320 0x7f5f980537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:39.731 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.719+0000 7f5fb4ff9700 1 -- 172.21.15.40:0/2767951368 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f5fb0069840 con 0x7f5fb80b6c80 2020-11-20T13:56:39.731 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.719+0000 7f5fb77fe700 1 --2- 172.21.15.40:0/2767951368 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f5f98051320 0x7f5f980537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:39.731 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.720+0000 7f5fb77fe700 1 --2- 172.21.15.40:0/2767951368 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f5f98051320 0x7f5f980537e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f5fa800d000 tx=0x7f5fa8007ac0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:39.732 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.723+0000 7f5fb4ff9700 1 -- 172.21.15.40:0/2767951368 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f5fb00419c0 con 0x7f5fb80b6c80 2020-11-20T13:56:39.886 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.885+0000 7f5fbdb84700 1 -- 172.21.15.40:0/2767951368 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5fb81534a0 con 0x7f5fb80b6c80 2020-11-20T13:56:39.887 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.885+0000 7f5fb4ff9700 1 -- 172.21.15.40:0/2767951368 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+10 (secure 0 0 0) 0x7f5fb0016020 con 0x7f5fb80b6c80 2020-11-20T13:56:39.887 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2020-11-20T13:56:39.888 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.887+0000 7f5fbdb84700 1 -- 172.21.15.40:0/2767951368 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f5f98051320 msgr2=0x7f5f980537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:39.888 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.887+0000 7f5fbdb84700 1 --2- 172.21.15.40:0/2767951368 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f5f98051320 0x7f5f980537e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f5fa800d000 tx=0x7f5fa8007ac0).stop 2020-11-20T13:56:39.889 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.887+0000 7f5fbdb84700 1 -- 172.21.15.40:0/2767951368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fb80b6c80 msgr2=0x7f5fb8063f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:39.889 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.887+0000 7f5fbdb84700 1 --2- 172.21.15.40:0/2767951368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fb80b6c80 0x7f5fb8063f00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f5fb000c000 tx=0x7f5fb00091a0).stop 2020-11-20T13:56:39.889 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.887+0000 7f5fbdb84700 1 -- 172.21.15.40:0/2767951368 shutdown_connections 2020-11-20T13:56:39.889 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.887+0000 7f5fbdb84700 1 --2- 172.21.15.40:0/2767951368 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f5f98051320 0x7f5f980537e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:39.890 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.887+0000 7f5fbdb84700 1 --2- 172.21.15.40:0/2767951368 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5fb80b1450 0x7f5fb80676b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:39.890 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.887+0000 7f5fbdb84700 1 --2- 172.21.15.40:0/2767951368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fb80b6c80 0x7f5fb8063f00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:39.890 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.887+0000 7f5fbdb84700 1 --2- 172.21.15.40:0/2767951368 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5fb80b31f0 0x7f5fb812bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:39.890 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.887+0000 7f5fbdb84700 1 -- 172.21.15.40:0/2767951368 >> 172.21.15.40:0/2767951368 conn(0x7f5fb80ba1d0 msgr2=0x7f5fb80b40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:39.890 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.887+0000 7f5fbdb84700 1 -- 172.21.15.40:0/2767951368 shutdown_connections 2020-11-20T13:56:39.891 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:39.887+0000 7f5fbdb84700 1 -- 172.21.15.40:0/2767951368 wait complete. 2020-11-20T13:56:39.899 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-20T13:56:40.020 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.016+0000 7f52af25d700 1 Processor -- start 2020-11-20T13:56:40.020 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.016+0000 7f52af25d700 1 -- start start 2020-11-20T13:56:40.020 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.016+0000 7f52af25d700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f52a80b5bf0 0x7f52a805f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:40.021 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.016+0000 7f52af25d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f52a8144390 0x7f52a805f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:40.021 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.016+0000 7f52af25d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f52a805fd70 0x7f52a80b2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:40.021 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.016+0000 7f52af25d700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52a80b5fd0 con 0x7f52a8144390 2020-11-20T13:56:40.021 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.017+0000 7f52af25d700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52a80b05f0 con 0x7f52a805fd70 2020-11-20T13:56:40.022 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.017+0000 7f52af25d700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52a8144870 con 0x7f52a80b5bf0 2020-11-20T13:56:40.022 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.017+0000 7f52a7fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f52a8144390 0x7f52a805f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:40.022 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.017+0000 7f52a7fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f52a8144390 0x7f52a805f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55128/0 (socket says 172.21.15.40:55128) 2020-11-20T13:56:40.022 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.017+0000 7f52a7fff700 1 -- 172.21.15.40:0/4061389780 learned_addr learned my addr 172.21.15.40:0/4061389780 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:40.022 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.017+0000 7f52acff9700 1 --2- 172.21.15.40:0/4061389780 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f52a80b5bf0 0x7f52a805f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:40.023 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.017+0000 7f52ad7fa700 1 --2- 172.21.15.40:0/4061389780 >> v2:172.21.15.116:3300/0 conn(0x7f52a805fd70 0x7f52a80b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:40.023 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.017+0000 7f52a7fff700 1 -- 172.21.15.40:0/4061389780 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f52a80b5bf0 msgr2=0x7f52a805f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:40.023 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.017+0000 7f52a7fff700 1 --2- 172.21.15.40:0/4061389780 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f52a80b5bf0 0x7f52a805f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:40.023 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.017+0000 7f52a7fff700 1 -- 172.21.15.40:0/4061389780 >> v2:172.21.15.116:3300/0 conn(0x7f52a805fd70 msgr2=0x7f52a80b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:40.024 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.017+0000 7f52a7fff700 1 --2- 172.21.15.40:0/4061389780 >> v2:172.21.15.116:3300/0 conn(0x7f52a805fd70 0x7f52a80b2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:40.024 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.017+0000 7f52a7fff700 1 -- 172.21.15.40:0/4061389780 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a80b0c30 con 0x7f52a8144390 2020-11-20T13:56:40.024 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.017+0000 7f52a7fff700 1 --2- 172.21.15.40:0/4061389780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52a8144390 0x7f52a805f880 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f52a80b2780 tx=0x7f5298006e80).ready entity=mon.0 client_cookie=9c484673a8f42562 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:40.024 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.017+0000 7f52a77fe700 1 -- 172.21.15.40:0/4061389780 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52980097b0 con 0x7f52a8144390 2020-11-20T13:56:40.024 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.017+0000 7f52a77fe700 1 -- 172.21.15.40:0/4061389780 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5298009910 con 0x7f52a8144390 2020-11-20T13:56:40.025 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.017+0000 7f52a77fe700 1 -- 172.21.15.40:0/4061389780 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5298009c70 con 0x7f52a8144390 2020-11-20T13:56:40.025 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.017+0000 7f52af25d700 1 -- 172.21.15.40:0/4061389780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52a8144390 msgr2=0x7f52a805f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:40.025 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.017+0000 7f52af25d700 1 --2- 172.21.15.40:0/4061389780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52a8144390 0x7f52a805f880 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f52a80b2780 tx=0x7f5298006e80).stop 2020-11-20T13:56:40.026 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.018+0000 7f52af25d700 1 -- 172.21.15.40:0/4061389780 shutdown_connections 2020-11-20T13:56:40.027 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.018+0000 7f52af25d700 1 --2- 172.21.15.40:0/4061389780 >> v2:172.21.15.116:3300/0 conn(0x7f52a805fd70 0x7f52a80b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:40.027 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.018+0000 7f52af25d700 1 --2- 172.21.15.40:0/4061389780 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f52a80b5bf0 0x7f52a805f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:40.028 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.018+0000 7f52af25d700 1 --2- 172.21.15.40:0/4061389780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52a8144390 0x7f52a805f880 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:40.028 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.018+0000 7f52af25d700 1 -- 172.21.15.40:0/4061389780 >> 172.21.15.40:0/4061389780 conn(0x7f52a80ba1d0 msgr2=0x7f52a80b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:40.028 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.018+0000 7f52af25d700 1 -- 172.21.15.40:0/4061389780 shutdown_connections 2020-11-20T13:56:40.028 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.018+0000 7f52af25d700 1 -- 172.21.15.40:0/4061389780 wait complete. 2020-11-20T13:56:40.028 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.019+0000 7f52af25d700 1 Processor -- start 2020-11-20T13:56:40.029 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.019+0000 7f52af25d700 1 -- start start 2020-11-20T13:56:40.032 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.020+0000 7f52af25d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52a805fd70 0x7f52a80ade70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:40.032 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.020+0000 7f52af25d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52a80b5bf0 0x7f52a813ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:40.032 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.020+0000 7f52af25d700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f52a8144390 0x7f52a8133a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:40.032 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.020+0000 7f52af25d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a813f200 con 0x7f52a80b5bf0 2020-11-20T13:56:40.033 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.020+0000 7f52af25d700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a813f490 con 0x7f52a805fd70 2020-11-20T13:56:40.033 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.020+0000 7f52af25d700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52a813f730 con 0x7f52a8144390 2020-11-20T13:56:40.033 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.020+0000 7f52a7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52a80b5bf0 0x7f52a813ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:40.033 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.020+0000 7f52a7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52a80b5bf0 0x7f52a813ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55134/0 (socket says 172.21.15.40:55134) 2020-11-20T13:56:40.033 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.020+0000 7f52a7fff700 1 -- 172.21.15.40:0/610267851 learned_addr learned my addr 172.21.15.40:0/610267851 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:40.034 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.020+0000 7f52acff9700 1 --2- 172.21.15.40:0/610267851 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52a805fd70 0x7f52a80ade70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:40.034 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.020+0000 7f52ad7fa700 1 --2- 172.21.15.40:0/610267851 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f52a8144390 0x7f52a8133a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:40.034 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.020+0000 7f52a7fff700 1 -- 172.21.15.40:0/610267851 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f52a8144390 msgr2=0x7f52a8133a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:40.034 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.020+0000 7f52a7fff700 1 --2- 172.21.15.40:0/610267851 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f52a8144390 0x7f52a8133a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:40.034 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.020+0000 7f52a7fff700 1 -- 172.21.15.40:0/610267851 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52a805fd70 msgr2=0x7f52a80ade70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:40.035 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.020+0000 7f52a7fff700 1 --2- 172.21.15.40:0/610267851 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52a805fd70 0x7f52a80ade70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:40.035 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.020+0000 7f52a7fff700 1 -- 172.21.15.40:0/610267851 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a8140da0 con 0x7f52a80b5bf0 2020-11-20T13:56:40.035 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.020+0000 7f52a7fff700 1 --2- 172.21.15.40:0/610267851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52a80b5bf0 0x7f52a813ed10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f52a80b2780 tx=0x7f5298015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:40.035 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.020+0000 7f52a5ffb700 1 -- 172.21.15.40:0/610267851 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5298011b40 con 0x7f52a80b5bf0 2020-11-20T13:56:40.036 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.020+0000 7f52a5ffb700 1 -- 172.21.15.40:0/610267851 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5298011ca0 con 0x7f52a80b5bf0 2020-11-20T13:56:40.036 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.020+0000 7f52af25d700 1 -- 172.21.15.40:0/610267851 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52a8136db0 con 0x7f52a80b5bf0 2020-11-20T13:56:40.036 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.020+0000 7f52a5ffb700 1 -- 172.21.15.40:0/610267851 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f529801e460 con 0x7f52a80b5bf0 2020-11-20T13:56:40.036 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.020+0000 7f52af25d700 1 -- 172.21.15.40:0/610267851 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52a81368d0 con 0x7f52a80b5bf0 2020-11-20T13:56:40.036 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.021+0000 7f52af25d700 1 -- 172.21.15.40:0/610267851 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52a8057050 con 0x7f52a80b5bf0 2020-11-20T13:56:40.037 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.022+0000 7f52a5ffb700 1 -- 172.21.15.40:0/610267851 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f529801a040 con 0x7f52a80b5bf0 2020-11-20T13:56:40.037 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.022+0000 7f52a5ffb700 1 --2- 172.21.15.40:0/610267851 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f528c0513c0 0x7f528c053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:40.037 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.022+0000 7f52a5ffb700 1 -- 172.21.15.40:0/610267851 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f529806a130 con 0x7f52a80b5bf0 2020-11-20T13:56:40.037 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.022+0000 7f52acff9700 1 --2- 172.21.15.40:0/610267851 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f528c0513c0 0x7f528c053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:40.037 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.022+0000 7f52acff9700 1 --2- 172.21.15.40:0/610267851 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f528c0513c0 0x7f528c053880 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f529000c000 tx=0x7f5290008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:40.038 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.025+0000 7f52a5ffb700 1 -- 172.21.15.40:0/610267851 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f5298042390 con 0x7f52a80b5bf0 2020-11-20T13:56:40.186 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.184+0000 7f52af25d700 1 -- 172.21.15.40:0/610267851 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f52a813f8a0 con 0x7f52a80b5bf0 2020-11-20T13:56:40.186 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.184+0000 7f52a5ffb700 1 -- 172.21.15.40:0/610267851 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+10 (secure 0 0 0) 0x7f5298023020 con 0x7f52a80b5bf0 2020-11-20T13:56:40.186 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2020-11-20T13:56:40.187 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.186+0000 7f52af25d700 1 -- 172.21.15.40:0/610267851 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f528c0513c0 msgr2=0x7f528c053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:40.188 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.186+0000 7f52af25d700 1 --2- 172.21.15.40:0/610267851 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f528c0513c0 0x7f528c053880 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f529000c000 tx=0x7f5290008040).stop 2020-11-20T13:56:40.188 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.186+0000 7f52af25d700 1 -- 172.21.15.40:0/610267851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52a80b5bf0 msgr2=0x7f52a813ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:40.189 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.186+0000 7f52af25d700 1 --2- 172.21.15.40:0/610267851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52a80b5bf0 0x7f52a813ed10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f52a80b2780 tx=0x7f5298015000).stop 2020-11-20T13:56:40.189 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.186+0000 7f52af25d700 1 -- 172.21.15.40:0/610267851 shutdown_connections 2020-11-20T13:56:40.189 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.186+0000 7f52af25d700 1 --2- 172.21.15.40:0/610267851 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f528c0513c0 0x7f528c053880 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:40.189 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.186+0000 7f52af25d700 1 --2- 172.21.15.40:0/610267851 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f52a8144390 0x7f52a8133a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:40.190 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.186+0000 7f52af25d700 1 --2- 172.21.15.40:0/610267851 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52a805fd70 0x7f52a80ade70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:40.190 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.186+0000 7f52af25d700 1 --2- 172.21.15.40:0/610267851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52a80b5bf0 0x7f52a813ed10 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:40.191 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.186+0000 7f52af25d700 1 -- 172.21.15.40:0/610267851 >> 172.21.15.40:0/610267851 conn(0x7f52a80ba1d0 msgr2=0x7f52a80b7650 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:40.191 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.186+0000 7f52af25d700 1 -- 172.21.15.40:0/610267851 shutdown_connections 2020-11-20T13:56:40.191 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.186+0000 7f52af25d700 1 -- 172.21.15.40:0/610267851 wait complete. 2020-11-20T13:56:40.197 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T13:56:40.198 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-20T13:56:40.313 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.311+0000 7f5c3b5d8700 1 Processor -- start 2020-11-20T13:56:40.313 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.312+0000 7f5c3b5d8700 1 -- start start 2020-11-20T13:56:40.315 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.312+0000 7f5c3b5d8700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5c340632c0 0x7f5c34147530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:40.315 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.312+0000 7f5c3b5d8700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c340b8100 con 0x7f5c34062ee0 2020-11-20T13:56:40.316 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.312+0000 7f5c3b5d8700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c340b83a0 con 0x7f5c3409fb80 2020-11-20T13:56:40.316 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.312+0000 7f5c3b5d8700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c34141980 con 0x7f5c340632c0 2020-11-20T13:56:40.316 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.313+0000 7f5c38b73700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5c340632c0 0x7f5c34147530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:40.316 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.313+0000 7f5c39374700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5c34062ee0 0x7f5c34141270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45736/0 (socket says 172.21.15.40:45736) 2020-11-20T13:56:40.316 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.313+0000 7f5c39374700 1 -- 172.21.15.40:0/1947938805 learned_addr learned my addr 172.21.15.40:0/1947938805 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:40.317 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.313+0000 7f5c38b73700 1 -- 172.21.15.40:0/1947938805 >> v1:172.21.15.116:6789/0 conn(0x7f5c3409fb80 legacy=0x7f5c340b5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:40.317 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.313+0000 7f5c38b73700 1 -- 172.21.15.40:0/1947938805 >> v1:172.21.15.40:6789/0 conn(0x7f5c34062ee0 legacy=0x7f5c34141270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:40.317 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.313+0000 7f5c23fff700 1 -- 172.21.15.40:0/1947938805 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 788387716 0 0) 0x7f5c340b8100 con 0x7f5c34062ee0 2020-11-20T13:56:40.317 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.313+0000 7f5c38b73700 1 -- 172.21.15.40:0/1947938805 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c340b7ae0 con 0x7f5c340632c0 2020-11-20T13:56:40.317 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.313+0000 7f5c38b73700 1 --2- 172.21.15.40:0/1947938805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c340632c0 0x7f5c34147530 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f5c30002f30 tx=0x7f5c30006dc0).ready entity=mon.0 client_cookie=e8aaaee398d8b90e server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:40.318 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.313+0000 7f5c23fff700 1 -- 172.21.15.40:0/1947938805 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c300097f0 con 0x7f5c340632c0 2020-11-20T13:56:40.319 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.313+0000 7f5c23fff700 1 -- 172.21.15.40:0/1947938805 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c30009950 con 0x7f5c340632c0 2020-11-20T13:56:40.319 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.313+0000 7f5c23fff700 1 -- 172.21.15.40:0/1947938805 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c30009cb0 con 0x7f5c340632c0 2020-11-20T13:56:40.319 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.313+0000 7f5c3b5d8700 1 -- 172.21.15.40:0/1947938805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c340632c0 msgr2=0x7f5c34147530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:40.319 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.313+0000 7f5c3b5d8700 1 --2- 172.21.15.40:0/1947938805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c340632c0 0x7f5c34147530 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f5c30002f30 tx=0x7f5c30006dc0).stop 2020-11-20T13:56:40.319 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.314+0000 7f5c3b5d8700 1 -- 172.21.15.40:0/1947938805 shutdown_connections 2020-11-20T13:56:40.320 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.314+0000 7f5c3b5d8700 1 --2- 172.21.15.40:0/1947938805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c340632c0 0x7f5c34147530 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:40.320 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.314+0000 7f5c3b5d8700 1 -- 172.21.15.40:0/1947938805 >> 172.21.15.40:0/1947938805 conn(0x7f5c340b9be0 msgr2=0x7f5c340b9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:40.320 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.314+0000 7f5c3b5d8700 1 -- 172.21.15.40:0/1947938805 shutdown_connections 2020-11-20T13:56:40.320 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.314+0000 7f5c3b5d8700 1 -- 172.21.15.40:0/1947938805 wait complete. 2020-11-20T13:56:40.324 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.315+0000 7f5c3b5d8700 1 Processor -- start 2020-11-20T13:56:40.324 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.315+0000 7f5c3b5d8700 1 -- start start 2020-11-20T13:56:40.324 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.315+0000 7f5c3b5d8700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5c34062ee0 0x7f5c3413f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:40.324 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.315+0000 7f5c3b5d8700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c340632c0 0x7f5c3413fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:40.325 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.315+0000 7f5c3b5d8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c3409fb80 0x7f5c34159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:40.325 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.315+0000 7f5c3b5d8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c34065c70 con 0x7f5c3409fb80 2020-11-20T13:56:40.325 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.315+0000 7f5c3b5d8700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3414b600 con 0x7f5c340632c0 2020-11-20T13:56:40.325 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.315+0000 7f5c3b5d8700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3414b8a0 con 0x7f5c34062ee0 2020-11-20T13:56:40.325 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.315+0000 7f5c39b75700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c3409fb80 0x7f5c34159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:40.326 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.316+0000 7f5c39b75700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c3409fb80 0x7f5c34159db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55148/0 (socket says 172.21.15.40:55148) 2020-11-20T13:56:40.326 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.316+0000 7f5c39b75700 1 -- 172.21.15.40:0/3939775804 learned_addr learned my addr 172.21.15.40:0/3939775804 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:40.326 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.316+0000 7f5c39374700 1 --2- 172.21.15.40:0/3939775804 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5c34062ee0 0x7f5c3413f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:40.326 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.316+0000 7f5c38b73700 1 --2- 172.21.15.40:0/3939775804 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c340632c0 0x7f5c3413fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:40.327 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.316+0000 7f5c39b75700 1 -- 172.21.15.40:0/3939775804 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5c34062ee0 msgr2=0x7f5c3413f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:40.327 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.316+0000 7f5c39b75700 1 --2- 172.21.15.40:0/3939775804 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5c34062ee0 0x7f5c3413f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:40.327 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.316+0000 7f5c39b75700 1 -- 172.21.15.40:0/3939775804 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c340632c0 msgr2=0x7f5c3413fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:40.327 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.316+0000 7f5c39b75700 1 --2- 172.21.15.40:0/3939775804 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c340632c0 0x7f5c3413fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:40.327 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.316+0000 7f5c39b75700 1 -- 172.21.15.40:0/3939775804 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c340b3760 con 0x7f5c3409fb80 2020-11-20T13:56:40.328 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.316+0000 7f5c39b75700 1 --2- 172.21.15.40:0/3939775804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c3409fb80 0x7f5c34159db0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f5c34069f10 tx=0x7f5c2c007e80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:40.328 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.316+0000 7f5c227fc700 1 -- 172.21.15.40:0/3939775804 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c2c00a5a0 con 0x7f5c3409fb80 2020-11-20T13:56:40.328 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.316+0000 7f5c227fc700 1 -- 172.21.15.40:0/3939775804 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c2c00a700 con 0x7f5c3409fb80 2020-11-20T13:56:40.328 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.316+0000 7f5c227fc700 1 -- 172.21.15.40:0/3939775804 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c2c00aa60 con 0x7f5c3409fb80 2020-11-20T13:56:40.329 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.316+0000 7f5c3b5d8700 1 -- 172.21.15.40:0/3939775804 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c340b39b0 con 0x7f5c3409fb80 2020-11-20T13:56:40.329 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.316+0000 7f5c3b5d8700 1 -- 172.21.15.40:0/3939775804 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c3415a460 con 0x7f5c3409fb80 2020-11-20T13:56:40.330 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.317+0000 7f5c3b5d8700 1 -- 172.21.15.40:0/3939775804 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c3414ba60 con 0x7f5c3409fb80 2020-11-20T13:56:40.330 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.317+0000 7f5c227fc700 1 -- 172.21.15.40:0/3939775804 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f5c2c019030 con 0x7f5c3409fb80 2020-11-20T13:56:40.330 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.318+0000 7f5c227fc700 1 --2- 172.21.15.40:0/3939775804 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f5c14051320 0x7f5c140537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:40.330 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.318+0000 7f5c227fc700 1 -- 172.21.15.40:0/3939775804 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f5c2c068990 con 0x7f5c3409fb80 2020-11-20T13:56:40.330 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.318+0000 7f5c39374700 1 --2- 172.21.15.40:0/3939775804 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f5c14051320 0x7f5c140537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:40.331 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.318+0000 7f5c39374700 1 --2- 172.21.15.40:0/3939775804 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f5c14051320 0x7f5c140537e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f5c2400d000 tx=0x7f5c24007910).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:40.331 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.321+0000 7f5c227fc700 1 -- 172.21.15.40:0/3939775804 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f5c2c040bb0 con 0x7f5c3409fb80 2020-11-20T13:56:40.538 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.536+0000 7f5c3b5d8700 1 -- 172.21.15.40:0/3939775804 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c34057050 con 0x7f5c3409fb80 2020-11-20T13:56:40.538 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.537+0000 7f5c227fc700 1 -- 172.21.15.40:0/3939775804 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+2134 (secure 0 0 0) 0x7f5c2c0441d0 con 0x7f5c3409fb80 2020-11-20T13:56:40.539 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:56:40.539 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/2019740031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2019740031},{"type":"v1","addr":"172.21.15.116:6835","nonce":2019740031}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T13:56:25.428178+0000","modified":"2020-11-20T13:56:27.432735+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6833/584131564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":584131564},{"type":"v1","addr":"172.21.15.40:6833","nonce":584131564}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-20T13:56:40.540 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.538+0000 7f5c3b5d8700 1 -- 172.21.15.40:0/3939775804 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f5c14051320 msgr2=0x7f5c140537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:40.540 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.538+0000 7f5c3b5d8700 1 --2- 172.21.15.40:0/3939775804 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f5c14051320 0x7f5c140537e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f5c2400d000 tx=0x7f5c24007910).stop 2020-11-20T13:56:40.540 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.538+0000 7f5c3b5d8700 1 -- 172.21.15.40:0/3939775804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c3409fb80 msgr2=0x7f5c34159db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:40.541 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.538+0000 7f5c3b5d8700 1 --2- 172.21.15.40:0/3939775804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c3409fb80 0x7f5c34159db0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f5c34069f10 tx=0x7f5c2c007e80).stop 2020-11-20T13:56:40.541 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.538+0000 7f5c3b5d8700 1 -- 172.21.15.40:0/3939775804 shutdown_connections 2020-11-20T13:56:40.542 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.538+0000 7f5c3b5d8700 1 --2- 172.21.15.40:0/3939775804 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f5c14051320 0x7f5c140537e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:40.543 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.538+0000 7f5c3b5d8700 1 --2- 172.21.15.40:0/3939775804 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5c34062ee0 0x7f5c3413f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:40.543 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.538+0000 7f5c3b5d8700 1 --2- 172.21.15.40:0/3939775804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c3409fb80 0x7f5c34159db0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:40.543 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.538+0000 7f5c3b5d8700 1 --2- 172.21.15.40:0/3939775804 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c340632c0 0x7f5c3413fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:40.543 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.538+0000 7f5c3b5d8700 1 -- 172.21.15.40:0/3939775804 >> 172.21.15.40:0/3939775804 conn(0x7f5c340b9be0 msgr2=0x7f5c34066f90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:40.543 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.539+0000 7f5c3b5d8700 1 -- 172.21.15.40:0/3939775804 shutdown_connections 2020-11-20T13:56:40.544 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:40.539+0000 7f5c3b5d8700 1 -- 172.21.15.40:0/3939775804 wait complete. 2020-11-20T13:56:40.544 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 7 2020-11-20T13:56:40.752 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T13:56:25.428178+0000', 'modified': '2020-11-20T13:56:27.432735+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4263}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4263': {'gid': 4263, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.40:6833/584131564', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6832', 'nonce': 584131564}, {'type': 'v1', 'addr': '172.21.15.40:6833', 'nonce': 584131564}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T13:56:40.752 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T13:56:40.753 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T13:56:40.889 INFO:teuthology.orchestra.run.smithi040.stdout:{"cluster_fsid":"da1ed737-5174-430f-a3c9-56f6bbfad13c","whoami":0,"id":4263,"want_state":"up:active","state":"up:active","rank_uptime":15.451539671000001,"mdsmap_epoch":7,"osdmap_epoch":24,"osdmap_epoch_barrier":24,"uptime":21.766084114000002} 2020-11-20T13:56:40.890 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "da1ed737-5174-430f-a3c9-56f6bbfad13c", "id": 4263, "mdsmap_epoch": 7, "osdmap_epoch": 24, "osdmap_epoch_barrier": 24, "rank_uptime": 15.451539671, "state": "up:active", "uptime": 21.766084114, "want_state": "up:active", "whoami": 0 } 2020-11-20T13:56:40.890 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-11-20T13:56:40.890 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-11-20T13:56:40.891 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi040.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2020-11-20T13:56:40.891 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:40.891 INFO:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2020-11-20T13:56:40.907 INFO:teuthology.orchestra.run.smithi040:> sudo modprobe fuse 2020-11-20T13:56:41.002 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-20T13:56:41.002 INFO:teuthology.orchestra.run.smithi040:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-20T13:56:41.034 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:41.035 INFO:teuthology.orchestra.run.smithi040:> ls /sys/fs/fuse/connections 2020-11-20T13:56:41.088 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-11-20T13:56:41.089 INFO:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2020-11-20T13:56:41.131 INFO:teuthology.orchestra.run.smithi040:> sudo modprobe fuse 2020-11-20T13:56:41.162 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-20T13:56:41.162 INFO:teuthology.orchestra.run.smithi040:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-20T13:56:41.210 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi040.stderr:2020-11-20T13:56:41.209+0000 7f8aa40191c0 -1 init, newargv = 0x562b93b76190 newargc=9 2020-11-20T13:56:41.210 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi040.stderr:ceph-fuse[26119]: starting ceph client 2020-11-20T13:56:41.215 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi040.stderr:ceph-fuse[26119]: starting fuse 2020-11-20T13:56:41.234 INFO:teuthology.orchestra.run.smithi040.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-11-20T13:56:41.235 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-20T13:56:41.236 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:41.236 INFO:teuthology.orchestra.run.smithi040:> ls /sys/fs/fuse/connections 2020-11-20T13:56:41.290 INFO:teuthology.orchestra.run.smithi040.stdout:47 2020-11-20T13:56:41.291 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2020-11-20T13:56:41.291 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:41.292 INFO:teuthology.orchestra.run.smithi040:> sudo python3 -c ' 2020-11-20T13:56:41.292 INFO:teuthology.orchestra.run.smithi040:> import glob 2020-11-20T13:56:41.292 INFO:teuthology.orchestra.run.smithi040:> import re 2020-11-20T13:56:41.293 INFO:teuthology.orchestra.run.smithi040:> import os 2020-11-20T13:56:41.293 INFO:teuthology.orchestra.run.smithi040:> import subprocess 2020-11-20T13:56:41.293 INFO:teuthology.orchestra.run.smithi040:> 2020-11-20T13:56:41.294 INFO:teuthology.orchestra.run.smithi040:> def find_socket(client_name): 2020-11-20T13:56:41.294 INFO:teuthology.orchestra.run.smithi040:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-11-20T13:56:41.294 INFO:teuthology.orchestra.run.smithi040:> files = glob.glob(asok_path) 2020-11-20T13:56:41.294 INFO:teuthology.orchestra.run.smithi040:> 2020-11-20T13:56:41.295 INFO:teuthology.orchestra.run.smithi040:> # Given a non-glob path, it better be there 2020-11-20T13:56:41.295 INFO:teuthology.orchestra.run.smithi040:> if "*" not in asok_path: 2020-11-20T13:56:41.295 INFO:teuthology.orchestra.run.smithi040:> assert(len(files) == 1) 2020-11-20T13:56:41.296 INFO:teuthology.orchestra.run.smithi040:> return files[0] 2020-11-20T13:56:41.296 INFO:teuthology.orchestra.run.smithi040:> 2020-11-20T13:56:41.296 INFO:teuthology.orchestra.run.smithi040:> for f in files: 2020-11-20T13:56:41.297 INFO:teuthology.orchestra.run.smithi040:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-11-20T13:56:41.297 INFO:teuthology.orchestra.run.smithi040:> if os.path.exists("/proc/{0}".format(pid)): 2020-11-20T13:56:41.297 INFO:teuthology.orchestra.run.smithi040:> return f 2020-11-20T13:56:41.298 INFO:teuthology.orchestra.run.smithi040:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-11-20T13:56:41.298 INFO:teuthology.orchestra.run.smithi040:> 2020-11-20T13:56:41.299 INFO:teuthology.orchestra.run.smithi040:> print(find_socket("client.0")) 2020-11-20T13:56:41.299 INFO:teuthology.orchestra.run.smithi040:> ' 2020-11-20T13:56:41.387 INFO:teuthology.orchestra.run.smithi040.stdout:/var/run/ceph/ceph-client.0.26119.asok 2020-11-20T13:56:41.391 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.26119.asok 2020-11-20T13:56:41.391 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:41.392 INFO:teuthology.orchestra.run.smithi040:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.26119.asok status 2020-11-20T13:56:41.503 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2020-11-20T13:56:41.504 INFO:teuthology.orchestra.run.smithi040.stdout: "metadata": { 2020-11-20T13:56:41.504 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_sha1": "a7b62340ad27b26642bb54ec916a97c7e1f07a34", 2020-11-20T13:56:41.504 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_version": "ceph version 15.2.5-588-ga7b62340 (a7b62340ad27b26642bb54ec916a97c7e1f07a34) octopus (stable)", 2020-11-20T13:56:41.504 INFO:teuthology.orchestra.run.smithi040.stdout: "entity_id": "0", 2020-11-20T13:56:41.505 INFO:teuthology.orchestra.run.smithi040.stdout: "hostname": "smithi040", 2020-11-20T13:56:41.505 INFO:teuthology.orchestra.run.smithi040.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-11-20T13:56:41.505 INFO:teuthology.orchestra.run.smithi040.stdout: "pid": "26119", 2020-11-20T13:56:41.505 INFO:teuthology.orchestra.run.smithi040.stdout: "root": "/" 2020-11-20T13:56:41.506 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2020-11-20T13:56:41.506 INFO:teuthology.orchestra.run.smithi040.stdout: "dentry_count": 0, 2020-11-20T13:56:41.506 INFO:teuthology.orchestra.run.smithi040.stdout: "dentry_pinned_count": 0, 2020-11-20T13:56:41.506 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4632, 2020-11-20T13:56:41.506 INFO:teuthology.orchestra.run.smithi040.stdout: "inst": { 2020-11-20T13:56:41.507 INFO:teuthology.orchestra.run.smithi040.stdout: "name": { 2020-11-20T13:56:41.507 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "client", 2020-11-20T13:56:41.507 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4632 2020-11-20T13:56:41.507 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2020-11-20T13:56:41.507 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": { 2020-11-20T13:56:41.508 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2020-11-20T13:56:41.508 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:0", 2020-11-20T13:56:41.508 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3332889729 2020-11-20T13:56:41.508 INFO:teuthology.orchestra.run.smithi040.stdout: } 2020-11-20T13:56:41.508 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2020-11-20T13:56:41.509 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": { 2020-11-20T13:56:41.509 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2020-11-20T13:56:41.509 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:0", 2020-11-20T13:56:41.509 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3332889729 2020-11-20T13:56:41.509 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2020-11-20T13:56:41.510 INFO:teuthology.orchestra.run.smithi040.stdout: "inst_str": "client.4632 172.21.15.40:0/3332889729", 2020-11-20T13:56:41.510 INFO:teuthology.orchestra.run.smithi040.stdout: "addr_str": "172.21.15.40:0/3332889729", 2020-11-20T13:56:41.510 INFO:teuthology.orchestra.run.smithi040.stdout: "inode_count": 1, 2020-11-20T13:56:41.510 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_epoch": 7, 2020-11-20T13:56:41.510 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_epoch": 24, 2020-11-20T13:56:41.511 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_epoch_barrier": 0, 2020-11-20T13:56:41.511 INFO:teuthology.orchestra.run.smithi040.stdout: "blacklisted": false 2020-11-20T13:56:41.511 INFO:teuthology.orchestra.run.smithi040.stdout:} 2020-11-20T13:56:41.513 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-20T13:56:41.627 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.626+0000 7fc797fff700 1 Processor -- start 2020-11-20T13:56:41.627 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.626+0000 7fc797fff700 1 -- start start 2020-11-20T13:56:41.631 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.627+0000 7fc797fff700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc7981484f0 0x7fc798144f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:41.631 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.627+0000 7fc797fff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc798145510 con 0x7fc7980b6b00 2020-11-20T13:56:41.631 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.627+0000 7fc797fff700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7981457b0 con 0x7fc79813f190 2020-11-20T13:56:41.631 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.627+0000 7fc797fff700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc798145a50 con 0x7fc7981484f0 2020-11-20T13:56:41.632 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.627+0000 7fc796ffd700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc7980b6b00 0x7fc79813ea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45762/0 (socket says 172.21.15.40:45762) 2020-11-20T13:56:41.632 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.627+0000 7fc796ffd700 1 -- 172.21.15.40:0/4273441176 learned_addr learned my addr 172.21.15.40:0/4273441176 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:41.632 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.627+0000 7fc7977fe700 1 --2- 172.21.15.40:0/4273441176 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc7981484f0 0x7fc798144f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:41.632 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.627+0000 7fc795ffb700 1 -- 172.21.15.40:0/4273441176 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3049266892 0 0) 0x7fc798145510 con 0x7fc7980b6b00 2020-11-20T13:56:41.633 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.627+0000 7fc795ffb700 1 -- 172.21.15.40:0/4273441176 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc778003230 con 0x7fc7980b6b00 2020-11-20T13:56:41.633 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.627+0000 7fc795ffb700 1 -- 172.21.15.40:0/4273441176 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3588699552 0 0) 0x7fc778003230 con 0x7fc7980b6b00 2020-11-20T13:56:41.633 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.627+0000 7fc795ffb700 1 -- 172.21.15.40:0/4273441176 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc7981484f0 msgr2=0x7fc798144f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:41.633 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.627+0000 7fc795ffb700 1 --2- 172.21.15.40:0/4273441176 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc7981484f0 0x7fc798144f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:41.633 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.627+0000 7fc795ffb700 1 -- 172.21.15.40:0/4273441176 >> v1:172.21.15.116:6789/0 conn(0x7fc79813f190 legacy=0x7fc7980af6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:41.634 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.627+0000 7fc795ffb700 1 -- 172.21.15.40:0/4273441176 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7980b6ee0 con 0x7fc7980b6b00 2020-11-20T13:56:41.634 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.627+0000 7fc795ffb700 1 -- 172.21.15.40:0/4273441176 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2366654341 0 0) 0x7fc790001a50 con 0x7fc7980b6b00 2020-11-20T13:56:41.634 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.627+0000 7fc795ffb700 1 -- 172.21.15.40:0/4273441176 >> v1:172.21.15.40:6789/0 conn(0x7fc7980b6b00 legacy=0x7fc79813ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:41.634 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.627+0000 7fc795ffb700 1 --2- 172.21.15.40:0/4273441176 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc778003750 0x7fc778003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:41.634 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.627+0000 7fc795ffb700 1 --2- 172.21.15.40:0/4273441176 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc7981484f0 0x7fc7780044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:41.635 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.627+0000 7fc795ffb700 1 --2- 172.21.15.40:0/4273441176 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc7780049c0 0x7fc778004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:41.635 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.627+0000 7fc795ffb700 1 -- 172.21.15.40:0/4273441176 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc798145a50 con 0x7fc778003750 2020-11-20T13:56:41.635 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.627+0000 7fc795ffb700 1 -- 172.21.15.40:0/4273441176 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc778004160 con 0x7fc7780049c0 2020-11-20T13:56:41.635 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.627+0000 7fc795ffb700 1 -- 172.21.15.40:0/4273441176 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc778004320 con 0x7fc7981484f0 2020-11-20T13:56:41.635 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.627+0000 7fc796ffd700 1 --2- 172.21.15.40:0/4273441176 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc778003750 0x7fc778003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:41.636 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.628+0000 7fc7977fe700 1 --2- 172.21.15.40:0/4273441176 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc7780049c0 0x7fc778004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:41.636 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.628+0000 7fc7967fc700 1 --2- 172.21.15.40:0/4273441176 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc7981484f0 0x7fc7780044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:41.636 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.628+0000 7fc796ffd700 1 -- 172.21.15.40:0/4273441176 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc7981484f0 msgr2=0x7fc7780044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:41.636 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.628+0000 7fc796ffd700 1 --2- 172.21.15.40:0/4273441176 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc7981484f0 0x7fc7780044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:41.636 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.628+0000 7fc796ffd700 1 -- 172.21.15.40:0/4273441176 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc7780049c0 msgr2=0x7fc778004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:41.637 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.628+0000 7fc796ffd700 1 --2- 172.21.15.40:0/4273441176 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc7780049c0 0x7fc778004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:41.637 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.628+0000 7fc796ffd700 1 -- 172.21.15.40:0/4273441176 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc778003230 con 0x7fc778003750 2020-11-20T13:56:41.637 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.628+0000 7fc796ffd700 1 --2- 172.21.15.40:0/4273441176 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc778003750 0x7fc778003b30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fc790001760 tx=0x7fc7900021d0).ready entity=mon.0 client_cookie=435b9de25a479504 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:41.637 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.628+0000 7fc795ffb700 1 -- 172.21.15.40:0/4273441176 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc790005b70 con 0x7fc778003750 2020-11-20T13:56:41.637 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.628+0000 7fc795ffb700 1 -- 172.21.15.40:0/4273441176 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc790005cd0 con 0x7fc778003750 2020-11-20T13:56:41.638 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.628+0000 7fc795ffb700 1 -- 172.21.15.40:0/4273441176 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7900072c0 con 0x7fc778003750 2020-11-20T13:56:41.638 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.628+0000 7fc797fff700 1 -- 172.21.15.40:0/4273441176 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc778003750 msgr2=0x7fc778003b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:41.638 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.628+0000 7fc797fff700 1 --2- 172.21.15.40:0/4273441176 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc778003750 0x7fc778003b30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fc790001760 tx=0x7fc7900021d0).stop 2020-11-20T13:56:41.638 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.628+0000 7fc7977fe700 1 -- 172.21.15.40:0/4273441176 reap_dead start 2020-11-20T13:56:41.638 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.629+0000 7fc797fff700 1 -- 172.21.15.40:0/4273441176 shutdown_connections 2020-11-20T13:56:41.639 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.629+0000 7fc797fff700 1 -- 172.21.15.40:0/4273441176 >> 172.21.15.40:0/4273441176 conn(0x7fc79805fe00 msgr2=0x7fc7980b6070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:41.639 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.629+0000 7fc797fff700 1 -- 172.21.15.40:0/4273441176 shutdown_connections 2020-11-20T13:56:41.639 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.629+0000 7fc797fff700 1 -- 172.21.15.40:0/4273441176 wait complete. 2020-11-20T13:56:41.639 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.630+0000 7fc797fff700 1 Processor -- start 2020-11-20T13:56:41.639 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.630+0000 7fc797fff700 1 -- start start 2020-11-20T13:56:41.647 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.630+0000 7fc797fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc7980b8360 0x7fc7981539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:41.648 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.630+0000 7fc797fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc798143710 0x7fc79812dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:41.648 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.630+0000 7fc797fff700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc79812e120 0x7fc7981445d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:41.648 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.630+0000 7fc797fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc798142600 con 0x7fc798143710 2020-11-20T13:56:41.648 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.630+0000 7fc797fff700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc798142360 con 0x7fc7980b8360 2020-11-20T13:56:41.649 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.630+0000 7fc797fff700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc798144130 con 0x7fc79812e120 2020-11-20T13:56:41.649 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.630+0000 7fc7967fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc798143710 0x7fc79812dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:41.649 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.630+0000 7fc7967fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc798143710 0x7fc79812dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55180/0 (socket says 172.21.15.40:55180) 2020-11-20T13:56:41.649 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.630+0000 7fc7967fc700 1 -- 172.21.15.40:0/3544594448 learned_addr learned my addr 172.21.15.40:0/3544594448 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:41.649 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.631+0000 7fc7977fe700 1 --2- 172.21.15.40:0/3544594448 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc79812e120 0x7fc7981445d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:41.650 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.631+0000 7fc796ffd700 1 --2- 172.21.15.40:0/3544594448 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc7980b8360 0x7fc7981539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:41.650 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.631+0000 7fc7967fc700 1 -- 172.21.15.40:0/3544594448 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc79812e120 msgr2=0x7fc7981445d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:41.651 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.631+0000 7fc7967fc700 1 --2- 172.21.15.40:0/3544594448 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc79812e120 0x7fc7981445d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:41.652 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.631+0000 7fc7967fc700 1 -- 172.21.15.40:0/3544594448 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc7980b8360 msgr2=0x7fc7981539f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:41.652 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.631+0000 7fc7967fc700 1 --2- 172.21.15.40:0/3544594448 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc7980b8360 0x7fc7981539f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:41.652 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.631+0000 7fc7967fc700 1 -- 172.21.15.40:0/3544594448 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7980b8740 con 0x7fc798143710 2020-11-20T13:56:41.652 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.631+0000 7fc796ffd700 1 --2- 172.21.15.40:0/3544594448 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc7980b8360 0x7fc7981539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:41.652 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.631+0000 7fc7967fc700 1 --2- 172.21.15.40:0/3544594448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc798143710 0x7fc79812dc30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fc78000d000 tx=0x7fc780007b40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:41.653 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.631+0000 7fc78ffff700 1 -- 172.21.15.40:0/3544594448 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc78000fc10 con 0x7fc798143710 2020-11-20T13:56:41.653 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.631+0000 7fc78ffff700 1 -- 172.21.15.40:0/3544594448 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc78000fd70 con 0x7fc798143710 2020-11-20T13:56:41.653 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.631+0000 7fc78ffff700 1 -- 172.21.15.40:0/3544594448 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc780014640 con 0x7fc798143710 2020-11-20T13:56:41.653 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.631+0000 7fc797fff700 1 -- 172.21.15.40:0/3544594448 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc798153ee0 con 0x7fc798143710 2020-11-20T13:56:41.653 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.631+0000 7fc797fff700 1 -- 172.21.15.40:0/3544594448 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc79813f570 con 0x7fc798143710 2020-11-20T13:56:41.654 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.632+0000 7fc797fff700 1 -- 172.21.15.40:0/3544594448 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc798142160 con 0x7fc798143710 2020-11-20T13:56:41.654 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.632+0000 7fc78ffff700 1 -- 172.21.15.40:0/3544594448 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fc7800133b0 con 0x7fc798143710 2020-11-20T13:56:41.654 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.632+0000 7fc78ffff700 1 --2- 172.21.15.40:0/3544594448 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fc77c0513c0 0x7fc77c053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:41.654 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.632+0000 7fc78ffff700 1 -- 172.21.15.40:0/3544594448 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fc780068fe0 con 0x7fc798143710 2020-11-20T13:56:41.654 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.633+0000 7fc796ffd700 1 --2- 172.21.15.40:0/3544594448 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fc77c0513c0 0x7fc77c053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:41.655 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.633+0000 7fc796ffd700 1 --2- 172.21.15.40:0/3544594448 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fc77c0513c0 0x7fc77c053880 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fc790014000 tx=0x7fc790015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:41.655 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.636+0000 7fc78ffff700 1 -- 172.21.15.40:0/3544594448 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fc7800411a0 con 0x7fc798143710 2020-11-20T13:56:41.799 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.798+0000 7fc797fff700 1 -- 172.21.15.40:0/3544594448 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc798057050 con 0x7fc798143710 2020-11-20T13:56:41.800 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.799+0000 7fc78ffff700 1 -- 172.21.15.40:0/3544594448 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16347 (secure 0 0 0) 0x7fc7800447c0 con 0x7fc798143710 2020-11-20T13:56:41.800 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:56:41.801 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":24,"fsid":"da1ed737-5174-430f-a3c9-56f6bbfad13c","created":"2020-11-20T13:56:04.990651+0000","modified":"2020-11-20T13:56:25.428032+0000","last_up_change":"2020-11-20T13:56:13.893026+0000","last_in_change":"2020-11-20T13:56:13.893026+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T13:56:08.330017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T13:56:17.939684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T13:56:20.563740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T13:56:23.062623+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9ea4c667-8938-46d6-a5ca-092d55c1f79e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":22825},{"type":"v1","addr":"172.21.15.40:6818","nonce":22825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":22825},{"type":"v1","addr":"172.21.15.40:6822","nonce":22825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":22825},{"type":"v1","addr":"172.21.15.40:6830","nonce":22825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":22825},{"type":"v1","addr":"172.21.15.40:6826","nonce":22825}]},"public_addr":"172.21.15.40:6818/22825","cluster_addr":"172.21.15.40:6822/22825","heartbeat_back_addr":"172.21.15.40:6830/22825","heartbeat_front_addr":"172.21.15.40:6826/22825","state":["exists","up"]},{"osd":1,"uuid":"515708d9-5087-4e55-8d14-cdea1c00eacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":22823},{"type":"v1","addr":"172.21.15.40:6801","nonce":22823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":22823},{"type":"v1","addr":"172.21.15.40:6803","nonce":22823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":22823},{"type":"v1","addr":"172.21.15.40:6807","nonce":22823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":22823},{"type":"v1","addr":"172.21.15.40:6805","nonce":22823}]},"public_addr":"172.21.15.40:6801/22823","cluster_addr":"172.21.15.40:6803/22823","heartbeat_back_addr":"172.21.15.40:6807/22823","heartbeat_front_addr":"172.21.15.40:6805/22823","state":["exists","up"]},{"osd":2,"uuid":"4d86bca7-743e-4983-b35f-36676c405b36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":22824},{"type":"v1","addr":"172.21.15.40:6809","nonce":22824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":22824},{"type":"v1","addr":"172.21.15.40:6811","nonce":22824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":22824},{"type":"v1","addr":"172.21.15.40:6815","nonce":22824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":22824},{"type":"v1","addr":"172.21.15.40:6813","nonce":22824}]},"public_addr":"172.21.15.40:6809/22824","cluster_addr":"172.21.15.40:6811/22824","heartbeat_back_addr":"172.21.15.40:6815/22824","heartbeat_front_addr":"172.21.15.40:6813/22824","state":["exists","up"]},{"osd":3,"uuid":"f4dc726b-6b34-4e8b-8eeb-87060b58002d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":22826},{"type":"v1","addr":"172.21.15.40:6819","nonce":22826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":22826},{"type":"v1","addr":"172.21.15.40:6823","nonce":22826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":22826},{"type":"v1","addr":"172.21.15.40:6831","nonce":22826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":22826},{"type":"v1","addr":"172.21.15.40:6827","nonce":22826}]},"public_addr":"172.21.15.40:6819/22826","cluster_addr":"172.21.15.40:6823/22826","heartbeat_back_addr":"172.21.15.40:6831/22826","heartbeat_front_addr":"172.21.15.40:6827/22826","state":["exists","up"]},{"osd":4,"uuid":"25a6d53b-d1f0-4498-b7d5-ffa38661d403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":23181},{"type":"v1","addr":"172.21.15.116:6819","nonce":23181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":23181},{"type":"v1","addr":"172.21.15.116:6821","nonce":23181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":23181},{"type":"v1","addr":"172.21.15.116:6825","nonce":23181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":23181},{"type":"v1","addr":"172.21.15.116:6823","nonce":23181}]},"public_addr":"172.21.15.116:6819/23181","cluster_addr":"172.21.15.116:6821/23181","heartbeat_back_addr":"172.21.15.116:6825/23181","heartbeat_front_addr":"172.21.15.116:6823/23181","state":["exists","up"]},{"osd":5,"uuid":"7d7a5af8-229b-4351-9a21-a3d509cd4635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":23171},{"type":"v1","addr":"172.21.15.116:6803","nonce":23171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":23171},{"type":"v1","addr":"172.21.15.116:6805","nonce":23171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":23171},{"type":"v1","addr":"172.21.15.116:6809","nonce":23171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":23171},{"type":"v1","addr":"172.21.15.116:6807","nonce":23171}]},"public_addr":"172.21.15.116:6803/23171","cluster_addr":"172.21.15.116:6805/23171","heartbeat_back_addr":"172.21.15.116:6809/23171","heartbeat_front_addr":"172.21.15.116:6807/23171","state":["exists","up"]},{"osd":6,"uuid":"8247d248-a140-4e36-9796-9847a97df58d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":23172},{"type":"v1","addr":"172.21.15.116:6811","nonce":23172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":23172},{"type":"v1","addr":"172.21.15.116:6813","nonce":23172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":23172},{"type":"v1","addr":"172.21.15.116:6817","nonce":23172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":23172},{"type":"v1","addr":"172.21.15.116:6815","nonce":23172}]},"public_addr":"172.21.15.116:6811/23172","cluster_addr":"172.21.15.116:6813/23172","heartbeat_back_addr":"172.21.15.116:6817/23172","heartbeat_front_addr":"172.21.15.116:6815/23172","state":["exists","up"]},{"osd":7,"uuid":"472803c4-70b8-499d-b18a-71aaeec6aab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":23196},{"type":"v1","addr":"172.21.15.116:6827","nonce":23196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":23196},{"type":"v1","addr":"172.21.15.116:6829","nonce":23196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":23196},{"type":"v1","addr":"172.21.15.116:6833","nonce":23196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":23196},{"type":"v1","addr":"172.21.15.116:6831","nonce":23196}]},"public_addr":"172.21.15.116:6827/23196","cluster_addr":"172.21.15.116:6829/23196","heartbeat_back_addr":"172.21.15.116:6833/23196","heartbeat_front_addr":"172.21.15.116:6831/23196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.029357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:11.989797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.091097+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.014766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.070860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.071864+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.063400+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.109028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-20T13:56:41.808 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.801+0000 7fc797fff700 1 -- 172.21.15.40:0/3544594448 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fc77c0513c0 msgr2=0x7fc77c053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:41.808 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.801+0000 7fc797fff700 1 --2- 172.21.15.40:0/3544594448 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fc77c0513c0 0x7fc77c053880 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fc790014000 tx=0x7fc790015000).stop 2020-11-20T13:56:41.808 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.801+0000 7fc797fff700 1 -- 172.21.15.40:0/3544594448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc798143710 msgr2=0x7fc79812dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:41.808 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.801+0000 7fc797fff700 1 --2- 172.21.15.40:0/3544594448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc798143710 0x7fc79812dc30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fc78000d000 tx=0x7fc780007b40).stop 2020-11-20T13:56:41.809 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.801+0000 7fc797fff700 1 -- 172.21.15.40:0/3544594448 shutdown_connections 2020-11-20T13:56:41.809 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.801+0000 7fc797fff700 1 --2- 172.21.15.40:0/3544594448 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fc77c0513c0 0x7fc77c053880 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:41.809 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.801+0000 7fc797fff700 1 --2- 172.21.15.40:0/3544594448 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc79812e120 0x7fc7981445d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:41.809 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.801+0000 7fc797fff700 1 --2- 172.21.15.40:0/3544594448 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc7980b8360 0x7fc7981539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:41.809 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.801+0000 7fc797fff700 1 --2- 172.21.15.40:0/3544594448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc798143710 0x7fc79812dc30 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:41.810 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.801+0000 7fc797fff700 1 -- 172.21.15.40:0/3544594448 >> 172.21.15.40:0/3544594448 conn(0x7fc79805fe00 msgr2=0x7fc7980acdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:41.810 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.801+0000 7fc797fff700 1 -- 172.21.15.40:0/3544594448 shutdown_connections 2020-11-20T13:56:41.810 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.801+0000 7fc797fff700 1 -- 172.21.15.40:0/3544594448 wait complete. 2020-11-20T13:56:41.812 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T13:56:08.330017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T13:56:17.939684+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T13:56:20.563740+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T13:56:23.062623+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T13:56:41.813 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-20T13:56:41.929 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.928+0000 7fcb73fdf700 1 Processor -- start 2020-11-20T13:56:41.930 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.928+0000 7fcb73fdf700 1 -- start start 2020-11-20T13:56:41.931 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.929+0000 7fcb73fdf700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcb6c0b6c80 0x7fcb6c0b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:41.932 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.929+0000 7fcb73fdf700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb6c0b0d10 con 0x7fcb6c0b1450 2020-11-20T13:56:41.932 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.929+0000 7fcb73fdf700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb6c0b0fb0 con 0x7fcb6c144bd0 2020-11-20T13:56:41.932 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.929+0000 7fcb73fdf700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb6c145060 con 0x7fcb6c0b6c80 2020-11-20T13:56:41.932 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.929+0000 7fcb71d7b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcb6c0b6c80 0x7fcb6c0b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:41.933 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.929+0000 7fcb7257c700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fcb6c0b1450 0x7fcb6c12ce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45786/0 (socket says 172.21.15.40:45786) 2020-11-20T13:56:41.933 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.929+0000 7fcb71d7b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcb6c0b6c80 0x7fcb6c0b31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55186/0 (socket says 172.21.15.40:55186) 2020-11-20T13:56:41.933 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.929+0000 7fcb7257c700 1 -- 172.21.15.40:0/2444145116 learned_addr learned my addr 172.21.15.40:0/2444145116 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:41.933 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.929+0000 7fcb71d7b700 1 -- 172.21.15.40:0/2444145116 >> v1:172.21.15.116:6789/0 conn(0x7fcb6c144bd0 legacy=0x7fcb6c0b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:41.933 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.929+0000 7fcb71d7b700 1 -- 172.21.15.40:0/2444145116 >> v1:172.21.15.40:6789/0 conn(0x7fcb6c0b1450 legacy=0x7fcb6c12ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:41.934 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.929+0000 7fcb71d7b700 1 -- 172.21.15.40:0/2444145116 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb6c0b7060 con 0x7fcb6c0b6c80 2020-11-20T13:56:41.934 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.929+0000 7fcb70d79700 1 -- 172.21.15.40:0/2444145116 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 655004100 0 0) 0x7fcb6c0b0d10 con 0x7fcb6c0b1450 2020-11-20T13:56:41.934 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.930+0000 7fcb71d7b700 1 --2- 172.21.15.40:0/2444145116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb6c0b6c80 0x7fcb6c0b31f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7fcb6c062590 tx=0x7fcb5c006d60).ready entity=mon.0 client_cookie=4844a36703764180 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:41.934 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.930+0000 7fcb70d79700 1 -- 172.21.15.40:0/2444145116 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb5c0097b0 con 0x7fcb6c0b6c80 2020-11-20T13:56:41.935 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.930+0000 7fcb70d79700 1 -- 172.21.15.40:0/2444145116 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5c009910 con 0x7fcb6c0b6c80 2020-11-20T13:56:41.935 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.930+0000 7fcb70d79700 1 -- 172.21.15.40:0/2444145116 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb5c009a80 con 0x7fcb6c0b6c80 2020-11-20T13:56:41.935 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.930+0000 7fcb73fdf700 1 -- 172.21.15.40:0/2444145116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb6c0b6c80 msgr2=0x7fcb6c0b31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:41.935 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.930+0000 7fcb73fdf700 1 --2- 172.21.15.40:0/2444145116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb6c0b6c80 0x7fcb6c0b31f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7fcb6c062590 tx=0x7fcb5c006d60).stop 2020-11-20T13:56:41.936 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.930+0000 7fcb73fdf700 1 -- 172.21.15.40:0/2444145116 shutdown_connections 2020-11-20T13:56:41.936 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.930+0000 7fcb73fdf700 1 --2- 172.21.15.40:0/2444145116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb6c0b6c80 0x7fcb6c0b31f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:41.936 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.930+0000 7fcb73fdf700 1 -- 172.21.15.40:0/2444145116 >> 172.21.15.40:0/2444145116 conn(0x7fcb6c0ba1d0 msgr2=0x7fcb6c0b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:41.936 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.930+0000 7fcb73fdf700 1 -- 172.21.15.40:0/2444145116 shutdown_connections 2020-11-20T13:56:41.936 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.930+0000 7fcb73fdf700 1 -- 172.21.15.40:0/2444145116 wait complete. 2020-11-20T13:56:41.939 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.931+0000 7fcb73fdf700 1 Processor -- start 2020-11-20T13:56:41.939 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.931+0000 7fcb73fdf700 1 -- start start 2020-11-20T13:56:41.939 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.932+0000 7fcb73fdf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb6c0b1450 0x7fcb6c1243f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:41.940 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.932+0000 7fcb73fdf700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcb6c0b6c80 0x7fcb6c062950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:41.940 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.932+0000 7fcb73fdf700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcb6c144bd0 0x7fcb6c062e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:41.940 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.932+0000 7fcb73fdf700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c0af420 con 0x7fcb6c0b1450 2020-11-20T13:56:41.940 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.932+0000 7fcb73fdf700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c0af6c0 con 0x7fcb6c0b6c80 2020-11-20T13:56:41.940 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.932+0000 7fcb73fdf700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c0af960 con 0x7fcb6c144bd0 2020-11-20T13:56:41.941 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.932+0000 7fcb71d7b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb6c0b1450 0x7fcb6c1243f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:41.941 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.932+0000 7fcb71d7b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb6c0b1450 0x7fcb6c1243f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55192/0 (socket says 172.21.15.40:55192) 2020-11-20T13:56:41.941 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.932+0000 7fcb71d7b700 1 -- 172.21.15.40:0/666887395 learned_addr learned my addr 172.21.15.40:0/666887395 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:41.941 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.932+0000 7fcb7257c700 1 --2- 172.21.15.40:0/666887395 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcb6c144bd0 0x7fcb6c062e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:41.942 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.932+0000 7fcb7157a700 1 --2- 172.21.15.40:0/666887395 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcb6c0b6c80 0x7fcb6c062950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:41.942 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.932+0000 7fcb71d7b700 1 -- 172.21.15.40:0/666887395 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcb6c144bd0 msgr2=0x7fcb6c062e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:41.942 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.932+0000 7fcb71d7b700 1 --2- 172.21.15.40:0/666887395 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcb6c144bd0 0x7fcb6c062e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:41.942 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.932+0000 7fcb71d7b700 1 -- 172.21.15.40:0/666887395 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcb6c0b6c80 msgr2=0x7fcb6c062950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:41.942 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.932+0000 7fcb71d7b700 1 --2- 172.21.15.40:0/666887395 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcb6c0b6c80 0x7fcb6c062950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:41.943 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.932+0000 7fcb71d7b700 1 -- 172.21.15.40:0/666887395 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb6c0afe00 con 0x7fcb6c0b1450 2020-11-20T13:56:41.943 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.932+0000 7fcb71d7b700 1 --2- 172.21.15.40:0/666887395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb6c0b1450 0x7fcb6c1243f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7fcb6c062590 tx=0x7fcb5c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:41.943 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.933+0000 7fcb5affd700 1 -- 172.21.15.40:0/666887395 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb5c021440 con 0x7fcb6c0b1450 2020-11-20T13:56:41.943 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.933+0000 7fcb5affd700 1 -- 172.21.15.40:0/666887395 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5c0215a0 con 0x7fcb6c0b1450 2020-11-20T13:56:41.943 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.933+0000 7fcb73fdf700 1 -- 172.21.15.40:0/666887395 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb6c0b0050 con 0x7fcb6c0b1450 2020-11-20T13:56:41.944 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.933+0000 7fcb5affd700 1 -- 172.21.15.40:0/666887395 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb5c021900 con 0x7fcb6c0b1450 2020-11-20T13:56:41.944 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.933+0000 7fcb73fdf700 1 -- 172.21.15.40:0/666887395 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb6c0aceb0 con 0x7fcb6c0b1450 2020-11-20T13:56:41.944 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.934+0000 7fcb5affd700 1 -- 172.21.15.40:0/666887395 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fcb5c027020 con 0x7fcb6c0b1450 2020-11-20T13:56:41.944 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.934+0000 7fcb73fdf700 1 -- 172.21.15.40:0/666887395 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb6c0afb20 con 0x7fcb6c0b1450 2020-11-20T13:56:41.944 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.934+0000 7fcb5affd700 1 --2- 172.21.15.40:0/666887395 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fcb4c051350 0x7fcb4c053810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:41.945 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.934+0000 7fcb5affd700 1 -- 172.21.15.40:0/666887395 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fcb5c06a200 con 0x7fcb6c0b1450 2020-11-20T13:56:41.945 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.934+0000 7fcb7157a700 1 --2- 172.21.15.40:0/666887395 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fcb4c051350 0x7fcb4c053810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:41.945 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.934+0000 7fcb7157a700 1 --2- 172.21.15.40:0/666887395 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fcb4c051350 0x7fcb4c053810 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fcb6800d000 tx=0x7fcb68007ad0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:41.945 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:41.938+0000 7fcb5affd700 1 -- 172.21.15.40:0/666887395 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fcb5c042420 con 0x7fcb6c0b1450 2020-11-20T13:56:42.097 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.096+0000 7fcb73fdf700 1 -- 172.21.15.40:0/666887395 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fcb6c057050 con 0x7fcb6c0b1450 2020-11-20T13:56:42.097 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.096+0000 7fcb5affd700 1 -- 172.21.15.40:0/666887395 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v24) v1 ==== 110+0+10 (secure 0 0 0) 0x7fcb5c015020 con 0x7fcb6c0b1450 2020-11-20T13:56:42.098 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2020-11-20T13:56:42.099 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.098+0000 7fcb73fdf700 1 -- 172.21.15.40:0/666887395 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fcb4c051350 msgr2=0x7fcb4c053810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:42.099 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.098+0000 7fcb73fdf700 1 --2- 172.21.15.40:0/666887395 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fcb4c051350 0x7fcb4c053810 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fcb6800d000 tx=0x7fcb68007ad0).stop 2020-11-20T13:56:42.100 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.098+0000 7fcb73fdf700 1 -- 172.21.15.40:0/666887395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb6c0b1450 msgr2=0x7fcb6c1243f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:42.100 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.098+0000 7fcb73fdf700 1 --2- 172.21.15.40:0/666887395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb6c0b1450 0x7fcb6c1243f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7fcb6c062590 tx=0x7fcb5c016000).stop 2020-11-20T13:56:42.100 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.098+0000 7fcb73fdf700 1 -- 172.21.15.40:0/666887395 shutdown_connections 2020-11-20T13:56:42.100 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.098+0000 7fcb73fdf700 1 --2- 172.21.15.40:0/666887395 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fcb4c051350 0x7fcb4c053810 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:42.101 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.098+0000 7fcb73fdf700 1 --2- 172.21.15.40:0/666887395 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcb6c144bd0 0x7fcb6c062e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:42.101 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.098+0000 7fcb73fdf700 1 --2- 172.21.15.40:0/666887395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb6c0b1450 0x7fcb6c1243f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:42.101 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.098+0000 7fcb73fdf700 1 --2- 172.21.15.40:0/666887395 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcb6c0b6c80 0x7fcb6c062950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:42.101 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.098+0000 7fcb73fdf700 1 -- 172.21.15.40:0/666887395 >> 172.21.15.40:0/666887395 conn(0x7fcb6c0ba1d0 msgr2=0x7fcb6c125cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:42.101 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.098+0000 7fcb73fdf700 1 -- 172.21.15.40:0/666887395 shutdown_connections 2020-11-20T13:56:42.102 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.098+0000 7fcb73fdf700 1 -- 172.21.15.40:0/666887395 wait complete. 2020-11-20T13:56:42.109 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-20T13:56:42.226 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.224+0000 7fec6d350700 1 Processor -- start 2020-11-20T13:56:42.226 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.224+0000 7fec6d350700 1 -- start start 2020-11-20T13:56:42.228 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.225+0000 7fec6d350700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fec680b6c80 0x7fec680b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:42.229 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.225+0000 7fec6d350700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fec680b1450 0x7fec680b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:42.230 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.225+0000 7fec6d350700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec680b0d10 con 0x7fec68144bd0 2020-11-20T13:56:42.230 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.225+0000 7fec6d350700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec68145060 con 0x7fec680b6c80 2020-11-20T13:56:42.230 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.225+0000 7fec6d350700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec680b0fb0 con 0x7fec680b1450 2020-11-20T13:56:42.230 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.225+0000 7fec66ffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fec680b6c80 0x7fec680b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:42.230 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.225+0000 7fec667fc700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fec68144bd0 0x7fec680b40f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45796/0 (socket says 172.21.15.40:45796) 2020-11-20T13:56:42.231 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.225+0000 7fec66ffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fec680b6c80 0x7fec680b31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55200/0 (socket says 172.21.15.40:55200) 2020-11-20T13:56:42.231 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.225+0000 7fec667fc700 1 -- 172.21.15.40:0/277692937 learned_addr learned my addr 172.21.15.40:0/277692937 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:42.231 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.226+0000 7fec677fe700 1 --2- 172.21.15.40:0/277692937 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fec680b1450 0x7fec680b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:42.231 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.226+0000 7fec66ffd700 1 -- 172.21.15.40:0/277692937 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fec680b1450 msgr2=0x7fec680b18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:42.232 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.226+0000 7fec66ffd700 1 --2- 172.21.15.40:0/277692937 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fec680b1450 0x7fec680b18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:42.232 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.226+0000 7fec66ffd700 1 -- 172.21.15.40:0/277692937 >> v1:172.21.15.40:6789/0 conn(0x7fec68144bd0 legacy=0x7fec680b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:42.232 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.226+0000 7fec66ffd700 1 -- 172.21.15.40:0/277692937 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec680b1da0 con 0x7fec680b6c80 2020-11-20T13:56:42.232 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.226+0000 7fec65ffb700 1 -- 172.21.15.40:0/277692937 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3304173963 0 0) 0x7fec680b0d10 con 0x7fec68144bd0 2020-11-20T13:56:42.233 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.226+0000 7fec66ffd700 1 --2- 172.21.15.40:0/277692937 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec680b6c80 0x7fec680b31f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7fec5c002f30 tx=0x7fec5c006d60).ready entity=mon.0 client_cookie=9d497ac9111507a1 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:42.233 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.226+0000 7fec65ffb700 1 -- 172.21.15.40:0/277692937 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec5c0097f0 con 0x7fec680b6c80 2020-11-20T13:56:42.233 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.226+0000 7fec65ffb700 1 -- 172.21.15.40:0/277692937 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec5c009950 con 0x7fec680b6c80 2020-11-20T13:56:42.233 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.226+0000 7fec65ffb700 1 -- 172.21.15.40:0/277692937 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec5c009ac0 con 0x7fec680b6c80 2020-11-20T13:56:42.233 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.226+0000 7fec6d350700 1 -- 172.21.15.40:0/277692937 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec680b6c80 msgr2=0x7fec680b31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:42.234 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.226+0000 7fec6d350700 1 --2- 172.21.15.40:0/277692937 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec680b6c80 0x7fec680b31f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7fec5c002f30 tx=0x7fec5c006d60).stop 2020-11-20T13:56:42.234 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.227+0000 7fec6d350700 1 -- 172.21.15.40:0/277692937 shutdown_connections 2020-11-20T13:56:42.235 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.227+0000 7fec6d350700 1 --2- 172.21.15.40:0/277692937 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fec680b1450 0x7fec680b18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:42.235 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.227+0000 7fec6d350700 1 --2- 172.21.15.40:0/277692937 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec680b6c80 0x7fec680b31f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:42.235 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.227+0000 7fec6d350700 1 -- 172.21.15.40:0/277692937 >> 172.21.15.40:0/277692937 conn(0x7fec680ba1d0 msgr2=0x7fec680b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:42.235 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.227+0000 7fec6d350700 1 -- 172.21.15.40:0/277692937 shutdown_connections 2020-11-20T13:56:42.236 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.227+0000 7fec6d350700 1 -- 172.21.15.40:0/277692937 wait complete. 2020-11-20T13:56:42.238 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.228+0000 7fec6d350700 1 Processor -- start 2020-11-20T13:56:42.239 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.228+0000 7fec6d350700 1 -- start start 2020-11-20T13:56:42.239 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.228+0000 7fec6d350700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fec680b1450 0x7fec68067290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:42.239 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.228+0000 7fec6d350700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec680b6c80 0x7fec68158280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:42.240 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.228+0000 7fec6d350700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec68144bd0 0x7fec6805f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:42.241 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.228+0000 7fec6d350700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6805ff90 con 0x7fec680b6c80 2020-11-20T13:56:42.241 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.228+0000 7fec6d350700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec681578a0 con 0x7fec68144bd0 2020-11-20T13:56:42.241 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.228+0000 7fec6d350700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec68157b40 con 0x7fec680b1450 2020-11-20T13:56:42.241 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.228+0000 7fec667fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec680b6c80 0x7fec68158280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:42.242 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.228+0000 7fec667fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec680b6c80 0x7fec68158280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55206/0 (socket says 172.21.15.40:55206) 2020-11-20T13:56:42.242 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.228+0000 7fec667fc700 1 -- 172.21.15.40:0/2361064671 learned_addr learned my addr 172.21.15.40:0/2361064671 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:42.242 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.228+0000 7fec66ffd700 1 --2- 172.21.15.40:0/2361064671 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fec680b1450 0x7fec68067290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:42.242 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.229+0000 7fec677fe700 1 --2- 172.21.15.40:0/2361064671 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec68144bd0 0x7fec6805f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:42.243 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.229+0000 7fec667fc700 1 -- 172.21.15.40:0/2361064671 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fec680b1450 msgr2=0x7fec68067290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:42.243 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.229+0000 7fec667fc700 1 --2- 172.21.15.40:0/2361064671 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fec680b1450 0x7fec68067290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:42.243 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.229+0000 7fec667fc700 1 -- 172.21.15.40:0/2361064671 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec68144bd0 msgr2=0x7fec6805f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:42.243 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.229+0000 7fec667fc700 1 --2- 172.21.15.40:0/2361064671 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec68144bd0 0x7fec6805f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:42.243 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.229+0000 7fec667fc700 1 -- 172.21.15.40:0/2361064671 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec68157de0 con 0x7fec680b6c80 2020-11-20T13:56:42.244 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.229+0000 7fec677fe700 1 --2- 172.21.15.40:0/2361064671 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec68144bd0 0x7fec6805f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:42.244 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.229+0000 7fec667fc700 1 --2- 172.21.15.40:0/2361064671 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec680b6c80 0x7fec68158280 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7fec58000e90 tx=0x7fec580079f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:42.244 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.229+0000 7fec57fff700 1 -- 172.21.15.40:0/2361064671 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec580072e0 con 0x7fec680b6c80 2020-11-20T13:56:42.244 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.229+0000 7fec57fff700 1 -- 172.21.15.40:0/2361064671 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec58007440 con 0x7fec680b6c80 2020-11-20T13:56:42.245 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.229+0000 7fec6d350700 1 -- 172.21.15.40:0/2361064671 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec6805fa40 con 0x7fec680b6c80 2020-11-20T13:56:42.245 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.229+0000 7fec57fff700 1 -- 172.21.15.40:0/2361064671 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec580057d0 con 0x7fec680b6c80 2020-11-20T13:56:42.245 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.229+0000 7fec6d350700 1 -- 172.21.15.40:0/2361064671 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec68065e10 con 0x7fec680b6c80 2020-11-20T13:56:42.246 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.230+0000 7fec6d350700 1 -- 172.21.15.40:0/2361064671 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec68157f70 con 0x7fec680b6c80 2020-11-20T13:56:42.246 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.230+0000 7fec57fff700 1 -- 172.21.15.40:0/2361064671 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fec58010070 con 0x7fec680b6c80 2020-11-20T13:56:42.246 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.231+0000 7fec57fff700 1 --2- 172.21.15.40:0/2361064671 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fec44051320 0x7fec440537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:42.246 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.231+0000 7fec57fff700 1 -- 172.21.15.40:0/2361064671 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fec580688c0 con 0x7fec680b6c80 2020-11-20T13:56:42.247 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.231+0000 7fec66ffd700 1 --2- 172.21.15.40:0/2361064671 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fec44051320 0x7fec440537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:42.247 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.231+0000 7fec66ffd700 1 --2- 172.21.15.40:0/2361064671 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fec44051320 0x7fec440537e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7fec5c01b000 tx=0x7fec5c016000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:42.247 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.234+0000 7fec57fff700 1 -- 172.21.15.40:0/2361064671 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fec58040a40 con 0x7fec680b6c80 2020-11-20T13:56:42.398 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.396+0000 7fec6d350700 1 -- 172.21.15.40:0/2361064671 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fec68057050 con 0x7fec680b6c80 2020-11-20T13:56:42.398 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.397+0000 7fec57fff700 1 -- 172.21.15.40:0/2361064671 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7fec58044060 con 0x7fec680b6c80 2020-11-20T13:56:42.398 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2020-11-20T13:56:42.399 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.398+0000 7fec6d350700 1 -- 172.21.15.40:0/2361064671 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fec44051320 msgr2=0x7fec440537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:42.400 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.398+0000 7fec6d350700 1 --2- 172.21.15.40:0/2361064671 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fec44051320 0x7fec440537e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7fec5c01b000 tx=0x7fec5c016000).stop 2020-11-20T13:56:42.400 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.398+0000 7fec6d350700 1 -- 172.21.15.40:0/2361064671 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec680b6c80 msgr2=0x7fec68158280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:42.400 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.398+0000 7fec6d350700 1 --2- 172.21.15.40:0/2361064671 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec680b6c80 0x7fec68158280 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7fec58000e90 tx=0x7fec580079f0).stop 2020-11-20T13:56:42.400 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.398+0000 7fec6d350700 1 -- 172.21.15.40:0/2361064671 shutdown_connections 2020-11-20T13:56:42.400 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.398+0000 7fec6d350700 1 --2- 172.21.15.40:0/2361064671 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fec44051320 0x7fec440537e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:42.401 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.398+0000 7fec6d350700 1 --2- 172.21.15.40:0/2361064671 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fec680b1450 0x7fec68067290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:42.401 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.398+0000 7fec6d350700 1 --2- 172.21.15.40:0/2361064671 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec68144bd0 0x7fec6805f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:42.401 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.398+0000 7fec6d350700 1 --2- 172.21.15.40:0/2361064671 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec680b6c80 0x7fec68158280 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:42.401 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.398+0000 7fec6d350700 1 -- 172.21.15.40:0/2361064671 >> 172.21.15.40:0/2361064671 conn(0x7fec680ba1d0 msgr2=0x7fec681451a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:42.402 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.398+0000 7fec6d350700 1 -- 172.21.15.40:0/2361064671 shutdown_connections 2020-11-20T13:56:42.402 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.398+0000 7fec6d350700 1 -- 172.21.15.40:0/2361064671 wait complete. 2020-11-20T13:56:42.412 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-20T13:56:42.531 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.529+0000 7feacbc79700 1 Processor -- start 2020-11-20T13:56:42.531 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.530+0000 7feacbc79700 1 -- start start 2020-11-20T13:56:42.534 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.530+0000 7feacbc79700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7feac40b31f0 0x7feac4144bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:42.534 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.530+0000 7feacbc79700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7feac40b1450 0x7feac40b1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:42.534 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.530+0000 7feacbc79700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feac40b1df0 con 0x7feac40b6c80 2020-11-20T13:56:42.535 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.530+0000 7feacbc79700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feac40b0d10 con 0x7feac40b1450 2020-11-20T13:56:42.535 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.530+0000 7feacbc79700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac40b0fb0 con 0x7feac40b31f0 2020-11-20T13:56:42.535 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.530+0000 7feaca216700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7feac40b1450 0x7feac40b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:42.535 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.531+0000 7feaca216700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7feac40b1450 0x7feac40b1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55212/0 (socket says 172.21.15.40:55212) 2020-11-20T13:56:42.536 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.531+0000 7feaca216700 1 -- 172.21.15.40:0/1165170469 learned_addr learned my addr 172.21.15.40:0/1165170469 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:42.536 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.531+0000 7feac9214700 1 --2- 172.21.15.40:0/1165170469 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7feac40b31f0 0x7feac4144bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:42.536 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.531+0000 7feaca216700 1 -- 172.21.15.40:0/1165170469 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7feac40b31f0 msgr2=0x7feac4144bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:42.536 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.531+0000 7feaca216700 1 --2- 172.21.15.40:0/1165170469 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7feac40b31f0 0x7feac4144bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:42.536 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.531+0000 7feaca216700 1 -- 172.21.15.40:0/1165170469 >> v1:172.21.15.116:6789/0 conn(0x7feac40b6c80 legacy=0x7feac40b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:42.537 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.531+0000 7feaca216700 1 -- 172.21.15.40:0/1165170469 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac40b7060 con 0x7feac40b1450 2020-11-20T13:56:42.537 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.531+0000 7feaca216700 1 --2- 172.21.15.40:0/1165170469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feac40b1450 0x7feac40b1900 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7feac40ba7a0 tx=0x7feabc006e80).ready entity=mon.0 client_cookie=4148b7da951b3e39 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:42.537 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.531+0000 7feac8a13700 1 -- 172.21.15.40:0/1165170469 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feabc009770 con 0x7feac40b1450 2020-11-20T13:56:42.537 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.531+0000 7feac8a13700 1 -- 172.21.15.40:0/1165170469 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feabc0098d0 con 0x7feac40b1450 2020-11-20T13:56:42.538 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.531+0000 7feac8a13700 1 -- 172.21.15.40:0/1165170469 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feabc009a40 con 0x7feac40b1450 2020-11-20T13:56:42.538 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.531+0000 7feacbc79700 1 -- 172.21.15.40:0/1165170469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feac40b1450 msgr2=0x7feac40b1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:42.538 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.531+0000 7feacbc79700 1 --2- 172.21.15.40:0/1165170469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feac40b1450 0x7feac40b1900 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7feac40ba7a0 tx=0x7feabc006e80).stop 2020-11-20T13:56:42.538 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.532+0000 7feacbc79700 1 -- 172.21.15.40:0/1165170469 shutdown_connections 2020-11-20T13:56:42.539 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.532+0000 7feacbc79700 1 --2- 172.21.15.40:0/1165170469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feac40b1450 0x7feac40b1900 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:42.539 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.532+0000 7feacbc79700 1 --2- 172.21.15.40:0/1165170469 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7feac40b31f0 0x7feac4144bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:42.539 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.532+0000 7feacbc79700 1 -- 172.21.15.40:0/1165170469 >> 172.21.15.40:0/1165170469 conn(0x7feac40ba1d0 msgr2=0x7feac40b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:42.540 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.532+0000 7feacbc79700 1 -- 172.21.15.40:0/1165170469 shutdown_connections 2020-11-20T13:56:42.540 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.532+0000 7feacbc79700 1 -- 172.21.15.40:0/1165170469 wait complete. 2020-11-20T13:56:42.540 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.533+0000 7feacbc79700 1 Processor -- start 2020-11-20T13:56:42.541 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.533+0000 7feacbc79700 1 -- start start 2020-11-20T13:56:42.545 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.533+0000 7feacbc79700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7feac40b1450 0x7feac4148040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:42.545 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.533+0000 7feacbc79700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feac40b31f0 0x7feac4157da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:42.545 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.533+0000 7feacbc79700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feac40b6c80 0x7feac405f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:42.546 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.533+0000 7feacbc79700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac40678c0 con 0x7feac40b31f0 2020-11-20T13:56:42.546 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.533+0000 7feacbc79700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac4143ba0 con 0x7feac40b6c80 2020-11-20T13:56:42.546 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.533+0000 7feacbc79700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac41543d0 con 0x7feac40b1450 2020-11-20T13:56:42.546 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.533+0000 7feac9214700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feac40b31f0 0x7feac4157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:42.549 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.533+0000 7feac9214700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feac40b31f0 0x7feac4157da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55222/0 (socket says 172.21.15.40:55222) 2020-11-20T13:56:42.549 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.533+0000 7feac9214700 1 -- 172.21.15.40:0/2950786445 learned_addr learned my addr 172.21.15.40:0/2950786445 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:42.549 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.534+0000 7feac9a15700 1 --2- 172.21.15.40:0/2950786445 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7feac40b1450 0x7feac4148040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:42.550 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.534+0000 7feaca216700 1 --2- 172.21.15.40:0/2950786445 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feac40b6c80 0x7feac405f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:42.550 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.534+0000 7feac9214700 1 -- 172.21.15.40:0/2950786445 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7feac40b1450 msgr2=0x7feac4148040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:42.550 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.534+0000 7feac9214700 1 --2- 172.21.15.40:0/2950786445 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7feac40b1450 0x7feac4148040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:42.550 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.534+0000 7feac9214700 1 -- 172.21.15.40:0/2950786445 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feac40b6c80 msgr2=0x7feac405f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:42.550 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.534+0000 7feac9214700 1 --2- 172.21.15.40:0/2950786445 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feac40b6c80 0x7feac405f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:42.551 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.534+0000 7feac9214700 1 -- 172.21.15.40:0/2950786445 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac4154670 con 0x7feac40b31f0 2020-11-20T13:56:42.551 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.534+0000 7feac9214700 1 --2- 172.21.15.40:0/2950786445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feac40b31f0 0x7feac4157da0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7feac000a000 tx=0x7feac0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:42.551 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.534+0000 7feab2ffd700 1 -- 172.21.15.40:0/2950786445 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feac000bb20 con 0x7feac40b31f0 2020-11-20T13:56:42.551 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.534+0000 7feab2ffd700 1 -- 172.21.15.40:0/2950786445 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feac000bc80 con 0x7feac40b31f0 2020-11-20T13:56:42.552 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.534+0000 7feacbc79700 1 -- 172.21.15.40:0/2950786445 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feac4154970 con 0x7feac40b31f0 2020-11-20T13:56:42.552 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.534+0000 7feab2ffd700 1 -- 172.21.15.40:0/2950786445 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feac0012420 con 0x7feac40b31f0 2020-11-20T13:56:42.552 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.534+0000 7feacbc79700 1 -- 172.21.15.40:0/2950786445 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feac4154db0 con 0x7feac40b31f0 2020-11-20T13:56:42.552 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.535+0000 7feab2ffd700 1 -- 172.21.15.40:0/2950786445 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7feac0012580 con 0x7feac40b31f0 2020-11-20T13:56:42.552 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.535+0000 7feacbc79700 1 -- 172.21.15.40:0/2950786445 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feac41541e0 con 0x7feac40b31f0 2020-11-20T13:56:42.553 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.535+0000 7feab2ffd700 1 --2- 172.21.15.40:0/2950786445 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7feaa4051320 0x7feaa40537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:42.553 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.535+0000 7feab2ffd700 1 -- 172.21.15.40:0/2950786445 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7feac0069830 con 0x7feac40b31f0 2020-11-20T13:56:42.553 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.536+0000 7feac9a15700 1 --2- 172.21.15.40:0/2950786445 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7feaa4051320 0x7feaa40537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:42.553 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.536+0000 7feac9a15700 1 --2- 172.21.15.40:0/2950786445 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7feaa4051320 0x7feaa40537e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7feab400b000 tx=0x7feab4009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:42.553 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.539+0000 7feab2ffd700 1 -- 172.21.15.40:0/2950786445 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7feac0041950 con 0x7feac40b31f0 2020-11-20T13:56:42.700 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.699+0000 7feacbc79700 1 -- 172.21.15.40:0/2950786445 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feac4057050 con 0x7feac40b31f0 2020-11-20T13:56:42.701 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.699+0000 7feab2ffd700 1 -- 172.21.15.40:0/2950786445 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+10 (secure 0 0 0) 0x7feac0044f70 con 0x7feac40b31f0 2020-11-20T13:56:42.701 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2020-11-20T13:56:42.702 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.700+0000 7feacbc79700 1 -- 172.21.15.40:0/2950786445 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7feaa4051320 msgr2=0x7feaa40537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:42.702 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.700+0000 7feacbc79700 1 --2- 172.21.15.40:0/2950786445 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7feaa4051320 0x7feaa40537e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7feab400b000 tx=0x7feab4009040).stop 2020-11-20T13:56:42.703 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.700+0000 7feacbc79700 1 -- 172.21.15.40:0/2950786445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feac40b31f0 msgr2=0x7feac4157da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:42.703 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.700+0000 7feacbc79700 1 --2- 172.21.15.40:0/2950786445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feac40b31f0 0x7feac4157da0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7feac000a000 tx=0x7feac0008040).stop 2020-11-20T13:56:42.703 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.701+0000 7feacbc79700 1 -- 172.21.15.40:0/2950786445 shutdown_connections 2020-11-20T13:56:42.703 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.701+0000 7feacbc79700 1 --2- 172.21.15.40:0/2950786445 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7feaa4051320 0x7feaa40537e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:42.703 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.701+0000 7feacbc79700 1 --2- 172.21.15.40:0/2950786445 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7feac40b1450 0x7feac4148040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:42.704 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.701+0000 7feacbc79700 1 --2- 172.21.15.40:0/2950786445 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feac40b6c80 0x7feac405f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:42.704 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.701+0000 7feacbc79700 1 --2- 172.21.15.40:0/2950786445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feac40b31f0 0x7feac4157da0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:42.704 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.701+0000 7feacbc79700 1 -- 172.21.15.40:0/2950786445 >> 172.21.15.40:0/2950786445 conn(0x7feac40ba1d0 msgr2=0x7feac41451a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:42.704 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.701+0000 7feacbc79700 1 -- 172.21.15.40:0/2950786445 shutdown_connections 2020-11-20T13:56:42.705 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.701+0000 7feacbc79700 1 -- 172.21.15.40:0/2950786445 wait complete. 2020-11-20T13:56:42.712 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-20T13:56:42.829 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.828+0000 7fae622a6700 1 Processor -- start 2020-11-20T13:56:42.829 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.828+0000 7fae622a6700 1 -- start start 2020-11-20T13:56:42.831 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.828+0000 7fae622a6700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fae5c0b31f0 0x7fae5c144bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:42.832 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.828+0000 7fae622a6700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fae5c0b1450 0x7fae5c0b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:42.832 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.828+0000 7fae622a6700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae5c0b1dd0 con 0x7fae5c0b6c80 2020-11-20T13:56:42.832 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.828+0000 7fae622a6700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae5c0b0d10 con 0x7fae5c0b1450 2020-11-20T13:56:42.832 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.828+0000 7fae622a6700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae5c0b0fb0 con 0x7fae5c0b31f0 2020-11-20T13:56:42.832 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.829+0000 7fae60843700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fae5c0b1450 0x7fae5c0b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:42.833 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.829+0000 7fae60843700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fae5c0b1450 0x7fae5c0b18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55230/0 (socket says 172.21.15.40:55230) 2020-11-20T13:56:42.833 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.829+0000 7fae60843700 1 -- 172.21.15.40:0/3555564260 learned_addr learned my addr 172.21.15.40:0/3555564260 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:42.833 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.829+0000 7fae5b7fe700 1 --2- 172.21.15.40:0/3555564260 >> v2:172.21.15.116:3300/0 conn(0x7fae5c0b31f0 0x7fae5c144bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:42.833 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.829+0000 7fae60843700 1 -- 172.21.15.40:0/3555564260 >> v2:172.21.15.116:3300/0 conn(0x7fae5c0b31f0 msgr2=0x7fae5c144bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:42.833 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.829+0000 7fae60843700 1 --2- 172.21.15.40:0/3555564260 >> v2:172.21.15.116:3300/0 conn(0x7fae5c0b31f0 0x7fae5c144bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:42.834 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.829+0000 7fae60843700 1 -- 172.21.15.40:0/3555564260 >> v1:172.21.15.116:6789/0 conn(0x7fae5c0b6c80 legacy=0x7fae5c0b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:42.834 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.829+0000 7fae60843700 1 -- 172.21.15.40:0/3555564260 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae5c0b7060 con 0x7fae5c0b1450 2020-11-20T13:56:42.834 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.829+0000 7fae60843700 1 --2- 172.21.15.40:0/3555564260 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae5c0b1450 0x7fae5c0b18e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7fae540032b0 tx=0x7fae54009270).ready entity=mon.0 client_cookie=19896f4b35ae2a2c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:42.834 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.829+0000 7fae5affd700 1 -- 172.21.15.40:0/3555564260 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae54009e60 con 0x7fae5c0b1450 2020-11-20T13:56:42.834 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.829+0000 7fae5affd700 1 -- 172.21.15.40:0/3555564260 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae5400bc00 con 0x7fae5c0b1450 2020-11-20T13:56:42.835 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.829+0000 7fae5affd700 1 -- 172.21.15.40:0/3555564260 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae5400bd70 con 0x7fae5c0b1450 2020-11-20T13:56:42.835 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.829+0000 7fae622a6700 1 -- 172.21.15.40:0/3555564260 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae5c0b1450 msgr2=0x7fae5c0b18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:42.835 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.829+0000 7fae622a6700 1 --2- 172.21.15.40:0/3555564260 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae5c0b1450 0x7fae5c0b18e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7fae540032b0 tx=0x7fae54009270).stop 2020-11-20T13:56:42.835 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.830+0000 7fae622a6700 1 -- 172.21.15.40:0/3555564260 shutdown_connections 2020-11-20T13:56:42.835 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.830+0000 7fae622a6700 1 --2- 172.21.15.40:0/3555564260 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae5c0b1450 0x7fae5c0b18e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:42.836 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.830+0000 7fae622a6700 1 --2- 172.21.15.40:0/3555564260 >> v2:172.21.15.116:3300/0 conn(0x7fae5c0b31f0 0x7fae5c144bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:42.836 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.830+0000 7fae622a6700 1 -- 172.21.15.40:0/3555564260 >> 172.21.15.40:0/3555564260 conn(0x7fae5c0ba1d0 msgr2=0x7fae5c0b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:42.836 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.830+0000 7fae622a6700 1 -- 172.21.15.40:0/3555564260 shutdown_connections 2020-11-20T13:56:42.836 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.830+0000 7fae622a6700 1 -- 172.21.15.40:0/3555564260 wait complete. 2020-11-20T13:56:42.840 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.831+0000 7fae622a6700 1 Processor -- start 2020-11-20T13:56:42.840 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.831+0000 7fae622a6700 1 -- start start 2020-11-20T13:56:42.841 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.831+0000 7fae622a6700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fae5c0b1450 0x7fae5c0b5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:42.841 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.831+0000 7fae622a6700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae5c0b31f0 0x7fae5c062a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:42.841 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.831+0000 7fae622a6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae5c0b6c80 0x7fae5c0accf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:42.841 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.832+0000 7fae622a6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c064eb0 con 0x7fae5c0b6c80 2020-11-20T13:56:42.842 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.832+0000 7fae622a6700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c062f70 con 0x7fae5c0b31f0 2020-11-20T13:56:42.842 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.832+0000 7fae622a6700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c065120 con 0x7fae5c0b1450 2020-11-20T13:56:42.842 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.832+0000 7fae60843700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae5c0b6c80 0x7fae5c0accf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:42.842 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.832+0000 7fae60843700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae5c0b6c80 0x7fae5c0accf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55236/0 (socket says 172.21.15.40:55236) 2020-11-20T13:56:42.842 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.832+0000 7fae60843700 1 -- 172.21.15.40:0/4072723181 learned_addr learned my addr 172.21.15.40:0/4072723181 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:42.843 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.832+0000 7fae5b7fe700 1 --2- 172.21.15.40:0/4072723181 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae5c0b31f0 0x7fae5c062a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:42.843 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.832+0000 7fae5bfff700 1 --2- 172.21.15.40:0/4072723181 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fae5c0b1450 0x7fae5c0b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:42.843 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.832+0000 7fae60843700 1 -- 172.21.15.40:0/4072723181 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fae5c0b1450 msgr2=0x7fae5c0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:42.843 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.832+0000 7fae60843700 1 --2- 172.21.15.40:0/4072723181 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fae5c0b1450 0x7fae5c0b5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:42.843 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.832+0000 7fae60843700 1 -- 172.21.15.40:0/4072723181 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae5c0b31f0 msgr2=0x7fae5c062a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:42.844 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.832+0000 7fae60843700 1 --2- 172.21.15.40:0/4072723181 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae5c0b31f0 0x7fae5c062a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:42.844 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.832+0000 7fae60843700 1 -- 172.21.15.40:0/4072723181 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae5c0653c0 con 0x7fae5c0b6c80 2020-11-20T13:56:42.844 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.832+0000 7fae5b7fe700 1 --2- 172.21.15.40:0/4072723181 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae5c0b31f0 0x7fae5c062a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:42.844 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.832+0000 7fae60843700 1 --2- 172.21.15.40:0/4072723181 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae5c0b6c80 0x7fae5c0accf0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7fae540032b0 tx=0x7fae54016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:42.844 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.832+0000 7fae597fa700 1 -- 172.21.15.40:0/4072723181 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae540105b0 con 0x7fae5c0b6c80 2020-11-20T13:56:42.845 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.832+0000 7fae597fa700 1 -- 172.21.15.40:0/4072723181 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae54010710 con 0x7fae5c0b6c80 2020-11-20T13:56:42.845 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.832+0000 7fae622a6700 1 -- 172.21.15.40:0/4072723181 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae5c1437b0 con 0x7fae5c0b6c80 2020-11-20T13:56:42.845 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.832+0000 7fae597fa700 1 -- 172.21.15.40:0/4072723181 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae54010a70 con 0x7fae5c0b6c80 2020-11-20T13:56:42.845 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.832+0000 7fae622a6700 1 -- 172.21.15.40:0/4072723181 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae5c0ad3a0 con 0x7fae5c0b6c80 2020-11-20T13:56:42.845 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.833+0000 7fae622a6700 1 -- 172.21.15.40:0/4072723181 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae5c065550 con 0x7fae5c0b6c80 2020-11-20T13:56:42.846 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.833+0000 7fae597fa700 1 -- 172.21.15.40:0/4072723181 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fae5400f070 con 0x7fae5c0b6c80 2020-11-20T13:56:42.846 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.834+0000 7fae597fa700 1 --2- 172.21.15.40:0/4072723181 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fae3c0513c0 0x7fae3c053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:42.846 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.834+0000 7fae597fa700 1 -- 172.21.15.40:0/4072723181 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fae5406a7a0 con 0x7fae5c0b6c80 2020-11-20T13:56:42.846 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.834+0000 7fae5bfff700 1 --2- 172.21.15.40:0/4072723181 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fae3c0513c0 0x7fae3c053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:42.846 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.834+0000 7fae5bfff700 1 --2- 172.21.15.40:0/4072723181 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fae3c0513c0 0x7fae3c053880 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fae4c00d000 tx=0x7fae4c007a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:42.847 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.837+0000 7fae597fa700 1 -- 172.21.15.40:0/4072723181 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fae54042a40 con 0x7fae5c0b6c80 2020-11-20T13:56:42.999 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.998+0000 7fae622a6700 1 -- 172.21.15.40:0/4072723181 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fae5c057050 con 0x7fae5c0b6c80 2020-11-20T13:56:43.000 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:42.999+0000 7fae597fa700 1 -- 172.21.15.40:0/4072723181 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+10 (secure 0 0 0) 0x7fae540150a0 con 0x7fae5c0b6c80 2020-11-20T13:56:43.000 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2020-11-20T13:56:43.001 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.000+0000 7fae622a6700 1 -- 172.21.15.40:0/4072723181 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fae3c0513c0 msgr2=0x7fae3c053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:43.002 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.000+0000 7fae622a6700 1 --2- 172.21.15.40:0/4072723181 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fae3c0513c0 0x7fae3c053880 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fae4c00d000 tx=0x7fae4c007a80).stop 2020-11-20T13:56:43.002 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.000+0000 7fae622a6700 1 -- 172.21.15.40:0/4072723181 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae5c0b6c80 msgr2=0x7fae5c0accf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:43.002 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.000+0000 7fae622a6700 1 --2- 172.21.15.40:0/4072723181 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae5c0b6c80 0x7fae5c0accf0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7fae540032b0 tx=0x7fae54016000).stop 2020-11-20T13:56:43.002 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.000+0000 7fae622a6700 1 -- 172.21.15.40:0/4072723181 shutdown_connections 2020-11-20T13:56:43.002 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.000+0000 7fae622a6700 1 --2- 172.21.15.40:0/4072723181 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fae3c0513c0 0x7fae3c053880 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:43.003 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.000+0000 7fae622a6700 1 --2- 172.21.15.40:0/4072723181 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fae5c0b1450 0x7fae5c0b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:43.003 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.000+0000 7fae622a6700 1 --2- 172.21.15.40:0/4072723181 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae5c0b6c80 0x7fae5c0accf0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:43.003 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.000+0000 7fae622a6700 1 --2- 172.21.15.40:0/4072723181 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae5c0b31f0 0x7fae5c062a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:43.003 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.000+0000 7fae622a6700 1 -- 172.21.15.40:0/4072723181 >> 172.21.15.40:0/4072723181 conn(0x7fae5c0ba1d0 msgr2=0x7fae5c125cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:43.003 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.000+0000 7fae622a6700 1 -- 172.21.15.40:0/4072723181 shutdown_connections 2020-11-20T13:56:43.004 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.000+0000 7fae622a6700 1 -- 172.21.15.40:0/4072723181 wait complete. 2020-11-20T13:56:43.012 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T13:56:43.013 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-20T13:56:43.127 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.125+0000 7f58aa207700 1 Processor -- start 2020-11-20T13:56:43.127 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.126+0000 7f58aa207700 1 -- start start 2020-11-20T13:56:43.130 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.126+0000 7f58aa207700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f58a409a280 0x7f58a40b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:43.130 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.126+0000 7f58aa207700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f58a409bc40 0x7f58a409c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:43.130 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.126+0000 7f58aa207700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58a40b6be0 con 0x7f58a40b88c0 2020-11-20T13:56:43.130 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.126+0000 7f58aa207700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58a4099000 con 0x7f58a409bc40 2020-11-20T13:56:43.131 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.126+0000 7f58aa207700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58a409c570 con 0x7f58a409a280 2020-11-20T13:56:43.131 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.126+0000 7f58a3fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f58a409bc40 0x7f58a409c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:43.131 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.126+0000 7f58a3fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f58a409bc40 0x7f58a409c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55244/0 (socket says 172.21.15.40:55244) 2020-11-20T13:56:43.131 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.126+0000 7f58a3fff700 1 -- 172.21.15.40:0/2168860872 learned_addr learned my addr 172.21.15.40:0/2168860872 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:43.131 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.127+0000 7f58a2ffd700 1 --2- 172.21.15.40:0/2168860872 >> v2:172.21.15.116:3300/0 conn(0x7f58a409a280 0x7f58a40b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:43.132 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.127+0000 7f58a3fff700 1 -- 172.21.15.40:0/2168860872 >> v2:172.21.15.116:3300/0 conn(0x7f58a409a280 msgr2=0x7f58a40b8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:43.132 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.127+0000 7f58a3fff700 1 --2- 172.21.15.40:0/2168860872 >> v2:172.21.15.116:3300/0 conn(0x7f58a409a280 0x7f58a40b8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:43.132 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.127+0000 7f58a3fff700 1 -- 172.21.15.40:0/2168860872 >> v1:172.21.15.116:6789/0 conn(0x7f58a40b88c0 legacy=0x7f58a409b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:43.132 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.127+0000 7f58a3fff700 1 -- 172.21.15.40:0/2168860872 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a40b4250 con 0x7f58a409bc40 2020-11-20T13:56:43.133 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.127+0000 7f58a3fff700 1 --2- 172.21.15.40:0/2168860872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58a409bc40 0x7f58a409c080 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f58a40b73b0 tx=0x7f589c009270).ready entity=mon.0 client_cookie=d386bcf497a725a3 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:43.133 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.127+0000 7f58a27fc700 1 -- 172.21.15.40:0/2168860872 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f589c00bbc0 con 0x7f58a409bc40 2020-11-20T13:56:43.133 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.127+0000 7f58a27fc700 1 -- 172.21.15.40:0/2168860872 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589c00bd20 con 0x7f58a409bc40 2020-11-20T13:56:43.133 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.127+0000 7f58a27fc700 1 -- 172.21.15.40:0/2168860872 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f589c011400 con 0x7f58a409bc40 2020-11-20T13:56:43.134 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.127+0000 7f58aa207700 1 -- 172.21.15.40:0/2168860872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58a409bc40 msgr2=0x7f58a409c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:43.134 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.127+0000 7f58aa207700 1 --2- 172.21.15.40:0/2168860872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58a409bc40 0x7f58a409c080 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f58a40b73b0 tx=0x7f589c009270).stop 2020-11-20T13:56:43.134 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.127+0000 7f58aa207700 1 -- 172.21.15.40:0/2168860872 shutdown_connections 2020-11-20T13:56:43.134 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.127+0000 7f58aa207700 1 --2- 172.21.15.40:0/2168860872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58a409bc40 0x7f58a409c080 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:43.134 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.127+0000 7f58aa207700 1 --2- 172.21.15.40:0/2168860872 >> v2:172.21.15.116:3300/0 conn(0x7f58a409a280 0x7f58a40b8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:43.135 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.128+0000 7f58aa207700 1 -- 172.21.15.40:0/2168860872 >> 172.21.15.40:0/2168860872 conn(0x7f58a40ba380 msgr2=0x7f58a40b6350 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:43.135 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.128+0000 7f58aa207700 1 -- 172.21.15.40:0/2168860872 shutdown_connections 2020-11-20T13:56:43.135 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.128+0000 7f58aa207700 1 -- 172.21.15.40:0/2168860872 wait complete. 2020-11-20T13:56:43.135 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.128+0000 7f58aa207700 1 Processor -- start 2020-11-20T13:56:43.136 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.129+0000 7f58aa207700 1 -- start start 2020-11-20T13:56:43.139 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.129+0000 7f58aa207700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f58a409a280 0x7f58a4067740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:43.140 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.129+0000 7f58aa207700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f58a409bc40 0x7f58a4138a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:43.140 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.129+0000 7f58aa207700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58a40b88c0 0x7f58a4138f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:43.140 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.129+0000 7f58aa207700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a4139440 con 0x7f58a40b88c0 2020-11-20T13:56:43.141 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.129+0000 7f58aa207700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a415a150 con 0x7f58a409a280 2020-11-20T13:56:43.141 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.129+0000 7f58aa207700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58a415a3f0 con 0x7f58a409bc40 2020-11-20T13:56:43.142 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.129+0000 7f58a3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58a40b88c0 0x7f58a4138f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:43.142 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.130+0000 7f58a3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58a40b88c0 0x7f58a4138f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55250/0 (socket says 172.21.15.40:55250) 2020-11-20T13:56:43.142 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.130+0000 7f58a3fff700 1 -- 172.21.15.40:0/966644214 learned_addr learned my addr 172.21.15.40:0/966644214 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:43.143 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.130+0000 7f58a2ffd700 1 --2- 172.21.15.40:0/966644214 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f58a409bc40 0x7f58a4138a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:43.143 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.130+0000 7f58a37fe700 1 --2- 172.21.15.40:0/966644214 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f58a409a280 0x7f58a4067740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:43.143 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.130+0000 7f58a3fff700 1 -- 172.21.15.40:0/966644214 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f58a409bc40 msgr2=0x7f58a4138a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:43.144 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.130+0000 7f58a3fff700 1 --2- 172.21.15.40:0/966644214 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f58a409bc40 0x7f58a4138a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:43.144 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.130+0000 7f58a3fff700 1 -- 172.21.15.40:0/966644214 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f58a409a280 msgr2=0x7f58a4067740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:43.144 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.130+0000 7f58a3fff700 1 --2- 172.21.15.40:0/966644214 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f58a409a280 0x7f58a4067740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:43.145 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.130+0000 7f58a3fff700 1 -- 172.21.15.40:0/966644214 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a415a690 con 0x7f58a40b88c0 2020-11-20T13:56:43.145 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.130+0000 7f58a2ffd700 1 --2- 172.21.15.40:0/966644214 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f58a409bc40 0x7f58a4138a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:43.145 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.130+0000 7f58a3fff700 1 --2- 172.21.15.40:0/966644214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58a40b88c0 0x7f58a4138f50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f58a40b73b0 tx=0x7f589c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:43.146 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.130+0000 7f58a0ff9700 1 -- 172.21.15.40:0/966644214 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f589c0105b0 con 0x7f58a40b88c0 2020-11-20T13:56:43.146 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.130+0000 7f58a0ff9700 1 -- 172.21.15.40:0/966644214 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589c010710 con 0x7f58a40b88c0 2020-11-20T13:56:43.146 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.130+0000 7f58a0ff9700 1 -- 172.21.15.40:0/966644214 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f589c010a70 con 0x7f58a40b88c0 2020-11-20T13:56:43.147 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.130+0000 7f58aa207700 1 -- 172.21.15.40:0/966644214 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a4159e40 con 0x7f58a40b88c0 2020-11-20T13:56:43.147 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.130+0000 7f58aa207700 1 -- 172.21.15.40:0/966644214 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58a409e7e0 con 0x7f58a40b88c0 2020-11-20T13:56:43.147 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.131+0000 7f58aa207700 1 -- 172.21.15.40:0/966644214 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58a415a820 con 0x7f58a40b88c0 2020-11-20T13:56:43.148 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.131+0000 7f58a0ff9700 1 -- 172.21.15.40:0/966644214 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f589c00f070 con 0x7f58a40b88c0 2020-11-20T13:56:43.148 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.132+0000 7f58a0ff9700 1 --2- 172.21.15.40:0/966644214 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f5884051320 0x7f58840537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:43.148 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.132+0000 7f58a0ff9700 1 -- 172.21.15.40:0/966644214 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f589c06a900 con 0x7f58a40b88c0 2020-11-20T13:56:43.149 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.132+0000 7f58a37fe700 1 --2- 172.21.15.40:0/966644214 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f5884051320 0x7f58840537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:43.149 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.132+0000 7f58a37fe700 1 --2- 172.21.15.40:0/966644214 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f5884051320 0x7f58840537e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f589400b000 tx=0x7f5894009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:43.150 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.135+0000 7f58a0ff9700 1 -- 172.21.15.40:0/966644214 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f589c042b60 con 0x7f58a40b88c0 2020-11-20T13:56:43.355 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.354+0000 7f58aa207700 1 -- 172.21.15.40:0/966644214 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58a415a820 con 0x7f58a40b88c0 2020-11-20T13:56:43.356 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.354+0000 7f58a0ff9700 1 -- 172.21.15.40:0/966644214 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+2134 (secure 0 0 0) 0x7f589c015090 con 0x7f58a40b88c0 2020-11-20T13:56:43.356 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:56:43.356 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/2019740031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2019740031},{"type":"v1","addr":"172.21.15.116:6835","nonce":2019740031}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T13:56:25.428178+0000","modified":"2020-11-20T13:56:27.432735+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6833/584131564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":584131564},{"type":"v1","addr":"172.21.15.40:6833","nonce":584131564}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-20T13:56:43.358 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.356+0000 7f58aa207700 1 -- 172.21.15.40:0/966644214 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f5884051320 msgr2=0x7f58840537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:43.358 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.356+0000 7f58aa207700 1 --2- 172.21.15.40:0/966644214 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f5884051320 0x7f58840537e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f589400b000 tx=0x7f5894009040).stop 2020-11-20T13:56:43.359 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.356+0000 7f58aa207700 1 -- 172.21.15.40:0/966644214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58a40b88c0 msgr2=0x7f58a4138f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:43.359 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.356+0000 7f58aa207700 1 --2- 172.21.15.40:0/966644214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58a40b88c0 0x7f58a4138f50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f58a40b73b0 tx=0x7f589c016000).stop 2020-11-20T13:56:43.359 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.356+0000 7f58aa207700 1 -- 172.21.15.40:0/966644214 shutdown_connections 2020-11-20T13:56:43.359 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.356+0000 7f58aa207700 1 --2- 172.21.15.40:0/966644214 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f5884051320 0x7f58840537e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:43.359 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.356+0000 7f58aa207700 1 --2- 172.21.15.40:0/966644214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58a40b88c0 0x7f58a4138f50 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:43.360 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.356+0000 7f58aa207700 1 --2- 172.21.15.40:0/966644214 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f58a409a280 0x7f58a4067740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:43.360 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.356+0000 7f58aa207700 1 --2- 172.21.15.40:0/966644214 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f58a409bc40 0x7f58a4138a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:43.360 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.356+0000 7f58aa207700 1 -- 172.21.15.40:0/966644214 >> 172.21.15.40:0/966644214 conn(0x7f58a40ba380 msgr2=0x7f58a40b4560 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:43.361 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.356+0000 7f58aa207700 1 -- 172.21.15.40:0/966644214 shutdown_connections 2020-11-20T13:56:43.361 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:43.356+0000 7f58aa207700 1 -- 172.21.15.40:0/966644214 wait complete. 2020-11-20T13:56:43.361 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 7 2020-11-20T13:56:43.568 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T13:56:25.428178+0000', 'modified': '2020-11-20T13:56:27.432735+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4263}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4263': {'gid': 4263, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.40:6833/584131564', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6832', 'nonce': 584131564}, {'type': 'v1', 'addr': '172.21.15.40:6833', 'nonce': 584131564}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T13:56:43.569 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T13:56:43.569 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T13:56:43.707 INFO:teuthology.orchestra.run.smithi040.stdout:{"cluster_fsid":"da1ed737-5174-430f-a3c9-56f6bbfad13c","whoami":0,"id":4263,"want_state":"up:active","state":"up:active","rank_uptime":18.269818373,"mdsmap_epoch":7,"osdmap_epoch":24,"osdmap_epoch_barrier":24,"uptime":24.584363031999999} 2020-11-20T13:56:43.708 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "da1ed737-5174-430f-a3c9-56f6bbfad13c", "id": 4263, "mdsmap_epoch": 7, "osdmap_epoch": 24, "osdmap_epoch_barrier": 24, "rank_uptime": 18.269818373, "state": "up:active", "uptime": 24.584363032, "want_state": "up:active", "whoami": 0 } 2020-11-20T13:56:43.708 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-11-20T13:56:43.709 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2020-11-20T13:56:43.709 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi040.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2020-11-20T13:56:43.709 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:43.710 INFO:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2020-11-20T13:56:43.724 INFO:teuthology.orchestra.run.smithi040:> sudo modprobe fuse 2020-11-20T13:56:43.795 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-20T13:56:43.797 INFO:teuthology.orchestra.run.smithi040:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-20T13:56:43.866 INFO:teuthology.orchestra.run.smithi040.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-11-20T13:56:43.867 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-20T13:56:43.867 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:43.868 INFO:teuthology.orchestra.run.smithi040:> ls /sys/fs/fuse/connections 2020-11-20T13:56:43.921 INFO:teuthology.orchestra.run.smithi040.stdout:47 2020-11-20T13:56:43.921 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [47] 2020-11-20T13:56:43.922 INFO:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2020-11-20T13:56:43.963 INFO:teuthology.orchestra.run.smithi040:> sudo modprobe fuse 2020-11-20T13:56:43.994 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-20T13:56:43.995 INFO:teuthology.orchestra.run.smithi040:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-20T13:56:44.043 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi040.stderr:2020-11-20T13:56:44.043+0000 7fda825de1c0 -1 init, newargv = 0x55b11d2b6580 newargc=9 2020-11-20T13:56:44.044 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi040.stderr:ceph-fuse[26601]: starting ceph client 2020-11-20T13:56:44.049 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi040.stderr:ceph-fuse[26601]: starting fuse 2020-11-20T13:56:44.070 INFO:teuthology.orchestra.run.smithi040.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-11-20T13:56:44.071 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-20T13:56:44.071 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:44.072 INFO:teuthology.orchestra.run.smithi040:> ls /sys/fs/fuse/connections 2020-11-20T13:56:44.125 INFO:teuthology.orchestra.run.smithi040.stdout:47 2020-11-20T13:56:44.126 INFO:teuthology.orchestra.run.smithi040.stdout:48 2020-11-20T13:56:44.126 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47, 48] 2020-11-20T13:56:44.126 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:44.127 INFO:teuthology.orchestra.run.smithi040:> sudo python3 -c ' 2020-11-20T13:56:44.127 INFO:teuthology.orchestra.run.smithi040:> import glob 2020-11-20T13:56:44.127 INFO:teuthology.orchestra.run.smithi040:> import re 2020-11-20T13:56:44.128 INFO:teuthology.orchestra.run.smithi040:> import os 2020-11-20T13:56:44.128 INFO:teuthology.orchestra.run.smithi040:> import subprocess 2020-11-20T13:56:44.128 INFO:teuthology.orchestra.run.smithi040:> 2020-11-20T13:56:44.129 INFO:teuthology.orchestra.run.smithi040:> def find_socket(client_name): 2020-11-20T13:56:44.129 INFO:teuthology.orchestra.run.smithi040:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2020-11-20T13:56:44.129 INFO:teuthology.orchestra.run.smithi040:> files = glob.glob(asok_path) 2020-11-20T13:56:44.130 INFO:teuthology.orchestra.run.smithi040:> 2020-11-20T13:56:44.130 INFO:teuthology.orchestra.run.smithi040:> # Given a non-glob path, it better be there 2020-11-20T13:56:44.131 INFO:teuthology.orchestra.run.smithi040:> if "*" not in asok_path: 2020-11-20T13:56:44.131 INFO:teuthology.orchestra.run.smithi040:> assert(len(files) == 1) 2020-11-20T13:56:44.131 INFO:teuthology.orchestra.run.smithi040:> return files[0] 2020-11-20T13:56:44.131 INFO:teuthology.orchestra.run.smithi040:> 2020-11-20T13:56:44.132 INFO:teuthology.orchestra.run.smithi040:> for f in files: 2020-11-20T13:56:44.132 INFO:teuthology.orchestra.run.smithi040:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-11-20T13:56:44.132 INFO:teuthology.orchestra.run.smithi040:> if os.path.exists("/proc/{0}".format(pid)): 2020-11-20T13:56:44.133 INFO:teuthology.orchestra.run.smithi040:> return f 2020-11-20T13:56:44.133 INFO:teuthology.orchestra.run.smithi040:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-11-20T13:56:44.133 INFO:teuthology.orchestra.run.smithi040:> 2020-11-20T13:56:44.134 INFO:teuthology.orchestra.run.smithi040:> print(find_socket("client.1")) 2020-11-20T13:56:44.134 INFO:teuthology.orchestra.run.smithi040:> ' 2020-11-20T13:56:44.221 INFO:teuthology.orchestra.run.smithi040.stdout:/var/run/ceph/ceph-client.1.26601.asok 2020-11-20T13:56:44.225 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.26601.asok 2020-11-20T13:56:44.225 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:44.226 INFO:teuthology.orchestra.run.smithi040:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.26601.asok status 2020-11-20T13:56:44.338 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2020-11-20T13:56:44.338 INFO:teuthology.orchestra.run.smithi040.stdout: "metadata": { 2020-11-20T13:56:44.339 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_sha1": "a7b62340ad27b26642bb54ec916a97c7e1f07a34", 2020-11-20T13:56:44.339 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_version": "ceph version 15.2.5-588-ga7b62340 (a7b62340ad27b26642bb54ec916a97c7e1f07a34) octopus (stable)", 2020-11-20T13:56:44.339 INFO:teuthology.orchestra.run.smithi040.stdout: "entity_id": "1", 2020-11-20T13:56:44.339 INFO:teuthology.orchestra.run.smithi040.stdout: "hostname": "smithi040", 2020-11-20T13:56:44.339 INFO:teuthology.orchestra.run.smithi040.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2020-11-20T13:56:44.340 INFO:teuthology.orchestra.run.smithi040.stdout: "pid": "26601", 2020-11-20T13:56:44.340 INFO:teuthology.orchestra.run.smithi040.stdout: "root": "/" 2020-11-20T13:56:44.340 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2020-11-20T13:56:44.340 INFO:teuthology.orchestra.run.smithi040.stdout: "dentry_count": 0, 2020-11-20T13:56:44.340 INFO:teuthology.orchestra.run.smithi040.stdout: "dentry_pinned_count": 0, 2020-11-20T13:56:44.341 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4683, 2020-11-20T13:56:44.341 INFO:teuthology.orchestra.run.smithi040.stdout: "inst": { 2020-11-20T13:56:44.341 INFO:teuthology.orchestra.run.smithi040.stdout: "name": { 2020-11-20T13:56:44.341 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "client", 2020-11-20T13:56:44.341 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4683 2020-11-20T13:56:44.342 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2020-11-20T13:56:44.342 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": { 2020-11-20T13:56:44.342 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2020-11-20T13:56:44.342 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:0", 2020-11-20T13:56:44.342 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3045147488 2020-11-20T13:56:44.343 INFO:teuthology.orchestra.run.smithi040.stdout: } 2020-11-20T13:56:44.343 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2020-11-20T13:56:44.343 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": { 2020-11-20T13:56:44.343 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2020-11-20T13:56:44.343 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:0", 2020-11-20T13:56:44.344 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3045147488 2020-11-20T13:56:44.344 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2020-11-20T13:56:44.344 INFO:teuthology.orchestra.run.smithi040.stdout: "inst_str": "client.4683 172.21.15.40:0/3045147488", 2020-11-20T13:56:44.344 INFO:teuthology.orchestra.run.smithi040.stdout: "addr_str": "172.21.15.40:0/3045147488", 2020-11-20T13:56:44.344 INFO:teuthology.orchestra.run.smithi040.stdout: "inode_count": 1, 2020-11-20T13:56:44.345 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_epoch": 7, 2020-11-20T13:56:44.345 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_epoch": 24, 2020-11-20T13:56:44.345 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_epoch_barrier": 0, 2020-11-20T13:56:44.345 INFO:teuthology.orchestra.run.smithi040.stdout: "blacklisted": false 2020-11-20T13:56:44.345 INFO:teuthology.orchestra.run.smithi040.stdout:} 2020-11-20T13:56:44.347 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-20T13:56:44.462 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.461+0000 7fafc63a3700 1 Processor -- start 2020-11-20T13:56:44.462 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.461+0000 7fafc63a3700 1 -- start start 2020-11-20T13:56:44.464 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.462+0000 7fafc63a3700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fafc0098f40 0x7fafc00af9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:44.465 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.462+0000 7fafc63a3700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fafc00ae810 0x7fafc0142500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:44.465 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.462+0000 7fafc63a3700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fafc00b4b90 0x7fafc01429f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:44.465 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.462+0000 7fafc63a3700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafc0099320 con 0x7fafc00b4b90 2020-11-20T13:56:44.466 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.462+0000 7fafc63a3700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafc00affd0 con 0x7fafc0098f40 2020-11-20T13:56:44.466 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.462+0000 7fafc63a3700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc00b0270 con 0x7fafc00ae810 2020-11-20T13:56:44.466 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.462+0000 7fafc4940700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fafc00b4b90 0x7fafc01429f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:44.466 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.462+0000 7fafc4940700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fafc00b4b90 0x7fafc01429f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55272/0 (socket says 172.21.15.40:55272) 2020-11-20T13:56:44.466 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.462+0000 7fafc4940700 1 -- 172.21.15.40:0/2034217562 learned_addr learned my addr 172.21.15.40:0/2034217562 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:44.467 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.462+0000 7fafbffff700 1 --2- 172.21.15.40:0/2034217562 >> v2:172.21.15.116:3300/0 conn(0x7fafc0098f40 0x7fafc00af9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:44.467 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.462+0000 7fafbf7fe700 1 --2- 172.21.15.40:0/2034217562 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fafc00ae810 0x7fafc0142500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:44.467 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.462+0000 7fafc4940700 1 -- 172.21.15.40:0/2034217562 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fafc00ae810 msgr2=0x7fafc0142500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:44.467 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.462+0000 7fafc4940700 1 --2- 172.21.15.40:0/2034217562 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fafc00ae810 0x7fafc0142500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:44.467 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.462+0000 7fafc4940700 1 -- 172.21.15.40:0/2034217562 >> v2:172.21.15.116:3300/0 conn(0x7fafc0098f40 msgr2=0x7fafc00af9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:44.468 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.462+0000 7fafc4940700 1 --2- 172.21.15.40:0/2034217562 >> v2:172.21.15.116:3300/0 conn(0x7fafc0098f40 0x7fafc00af9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:44.468 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.462+0000 7fafc4940700 1 -- 172.21.15.40:0/2034217562 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc00b0510 con 0x7fafc00b4b90 2020-11-20T13:56:44.468 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.462+0000 7fafc4940700 1 --2- 172.21.15.40:0/2034217562 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafc00b4b90 0x7fafc01429f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7fafb80032b0 tx=0x7fafb8009270).ready entity=mon.0 client_cookie=3fc040e0cec6a8bb server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:44.468 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.462+0000 7fafbeffd700 1 -- 172.21.15.40:0/2034217562 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafb8009e60 con 0x7fafc00b4b90 2020-11-20T13:56:44.468 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.462+0000 7fafbeffd700 1 -- 172.21.15.40:0/2034217562 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb800bb50 con 0x7fafc00b4b90 2020-11-20T13:56:44.469 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.462+0000 7fafbeffd700 1 -- 172.21.15.40:0/2034217562 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafb800bcc0 con 0x7fafc00b4b90 2020-11-20T13:56:44.469 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.463+0000 7fafc63a3700 1 -- 172.21.15.40:0/2034217562 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafc00b4b90 msgr2=0x7fafc01429f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:44.469 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.463+0000 7fafc63a3700 1 --2- 172.21.15.40:0/2034217562 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafc00b4b90 0x7fafc01429f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7fafb80032b0 tx=0x7fafb8009270).stop 2020-11-20T13:56:44.469 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.463+0000 7fafc63a3700 1 -- 172.21.15.40:0/2034217562 shutdown_connections 2020-11-20T13:56:44.469 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.463+0000 7fafc63a3700 1 --2- 172.21.15.40:0/2034217562 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafc00b4b90 0x7fafc01429f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:44.470 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.463+0000 7fafc63a3700 1 --2- 172.21.15.40:0/2034217562 >> v2:172.21.15.116:3300/0 conn(0x7fafc0098f40 0x7fafc00af9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:44.470 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.463+0000 7fafc63a3700 1 --2- 172.21.15.40:0/2034217562 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fafc00ae810 0x7fafc0142500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:44.470 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.463+0000 7fafc63a3700 1 -- 172.21.15.40:0/2034217562 >> 172.21.15.40:0/2034217562 conn(0x7fafc00ba380 msgr2=0x7fafc00b6350 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:44.470 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.463+0000 7fafc63a3700 1 -- 172.21.15.40:0/2034217562 shutdown_connections 2020-11-20T13:56:44.470 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.463+0000 7fafc63a3700 1 -- 172.21.15.40:0/2034217562 wait complete. 2020-11-20T13:56:44.476 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.464+0000 7fafc63a3700 1 Processor -- start 2020-11-20T13:56:44.476 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.464+0000 7fafc63a3700 1 -- start start 2020-11-20T13:56:44.476 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.465+0000 7fafc63a3700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fafc0098f40 0x7fafc0146c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:44.476 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.465+0000 7fafc63a3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafc00ae810 0x7fafc00b8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:44.476 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.465+0000 7fafc63a3700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fafc00b4b90 0x7fafc01404e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:44.478 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.465+0000 7fafc63a3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc00998b0 con 0x7fafc00ae810 2020-11-20T13:56:44.478 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.465+0000 7fafc63a3700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc0099b50 con 0x7fafc00b4b90 2020-11-20T13:56:44.478 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.465+0000 7fafc63a3700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc0099df0 con 0x7fafc0098f40 2020-11-20T13:56:44.478 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.465+0000 7fafbf7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafc00ae810 0x7fafc00b8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:44.479 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.465+0000 7fafbf7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafc00ae810 0x7fafc00b8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55276/0 (socket says 172.21.15.40:55276) 2020-11-20T13:56:44.479 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.465+0000 7fafbf7fe700 1 -- 172.21.15.40:0/1704568923 learned_addr learned my addr 172.21.15.40:0/1704568923 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:44.479 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.465+0000 7fafbffff700 1 --2- 172.21.15.40:0/1704568923 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fafc0098f40 0x7fafc0146c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:44.479 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.465+0000 7fafc4940700 1 --2- 172.21.15.40:0/1704568923 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fafc00b4b90 0x7fafc01404e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:44.479 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.465+0000 7fafbf7fe700 1 -- 172.21.15.40:0/1704568923 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fafc0098f40 msgr2=0x7fafc0146c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:44.480 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.465+0000 7fafbf7fe700 1 --2- 172.21.15.40:0/1704568923 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fafc0098f40 0x7fafc0146c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:44.480 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.465+0000 7fafbf7fe700 1 -- 172.21.15.40:0/1704568923 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fafc00b4b90 msgr2=0x7fafc01404e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:44.480 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.465+0000 7fafbf7fe700 1 --2- 172.21.15.40:0/1704568923 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fafc00b4b90 0x7fafc01404e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:44.480 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.465+0000 7fafbf7fe700 1 -- 172.21.15.40:0/1704568923 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc0148090 con 0x7fafc00ae810 2020-11-20T13:56:44.480 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.465+0000 7fafbf7fe700 1 --2- 172.21.15.40:0/1704568923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafc00ae810 0x7fafc00b8da0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7fafb400c000 tx=0x7fafb4006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:44.481 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.466+0000 7fafbd7fa700 1 -- 172.21.15.40:0/1704568923 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafb400ca10 con 0x7fafc00ae810 2020-11-20T13:56:44.481 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.466+0000 7fafbd7fa700 1 -- 172.21.15.40:0/1704568923 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb400cb70 con 0x7fafc00ae810 2020-11-20T13:56:44.481 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.466+0000 7fafc63a3700 1 -- 172.21.15.40:0/1704568923 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafc00b0c00 con 0x7fafc00ae810 2020-11-20T13:56:44.481 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.466+0000 7fafbd7fa700 1 -- 172.21.15.40:0/1704568923 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafb40123f0 con 0x7fafc00ae810 2020-11-20T13:56:44.482 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.466+0000 7fafc63a3700 1 -- 172.21.15.40:0/1704568923 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc0147de0 con 0x7fafc00ae810 2020-11-20T13:56:44.482 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.467+0000 7fafc63a3700 1 -- 172.21.15.40:0/1704568923 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafc0099fb0 con 0x7fafc00ae810 2020-11-20T13:56:44.482 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.467+0000 7fafbd7fa700 1 -- 172.21.15.40:0/1704568923 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fafb4012550 con 0x7fafc00ae810 2020-11-20T13:56:44.482 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.467+0000 7fafbd7fa700 1 --2- 172.21.15.40:0/1704568923 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fafa0051210 0x7fafa00536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:44.482 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.467+0000 7fafbd7fa700 1 -- 172.21.15.40:0/1704568923 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fafb40694e0 con 0x7fafc00ae810 2020-11-20T13:56:44.482 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.468+0000 7fafbffff700 1 --2- 172.21.15.40:0/1704568923 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fafa0051210 0x7fafa00536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:44.483 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.468+0000 7fafbffff700 1 --2- 172.21.15.40:0/1704568923 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fafa0051210 0x7fafa00536d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7fafb000b000 tx=0x7fafb0006dc0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:44.483 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.471+0000 7fafbd7fa700 1 -- 172.21.15.40:0/1704568923 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fafb40416a0 con 0x7fafc00ae810 2020-11-20T13:56:44.631 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.630+0000 7fafc63a3700 1 -- 172.21.15.40:0/1704568923 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fafc0057050 con 0x7fafc00ae810 2020-11-20T13:56:44.632 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.631+0000 7fafbd7fa700 1 -- 172.21.15.40:0/1704568923 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16347 (secure 0 0 0) 0x7fafb4044cc0 con 0x7fafc00ae810 2020-11-20T13:56:44.632 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:56:44.632 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":24,"fsid":"da1ed737-5174-430f-a3c9-56f6bbfad13c","created":"2020-11-20T13:56:04.990651+0000","modified":"2020-11-20T13:56:25.428032+0000","last_up_change":"2020-11-20T13:56:13.893026+0000","last_in_change":"2020-11-20T13:56:13.893026+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T13:56:08.330017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T13:56:17.939684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T13:56:20.563740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T13:56:23.062623+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9ea4c667-8938-46d6-a5ca-092d55c1f79e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":22825},{"type":"v1","addr":"172.21.15.40:6818","nonce":22825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":22825},{"type":"v1","addr":"172.21.15.40:6822","nonce":22825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":22825},{"type":"v1","addr":"172.21.15.40:6830","nonce":22825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":22825},{"type":"v1","addr":"172.21.15.40:6826","nonce":22825}]},"public_addr":"172.21.15.40:6818/22825","cluster_addr":"172.21.15.40:6822/22825","heartbeat_back_addr":"172.21.15.40:6830/22825","heartbeat_front_addr":"172.21.15.40:6826/22825","state":["exists","up"]},{"osd":1,"uuid":"515708d9-5087-4e55-8d14-cdea1c00eacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":22823},{"type":"v1","addr":"172.21.15.40:6801","nonce":22823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":22823},{"type":"v1","addr":"172.21.15.40:6803","nonce":22823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":22823},{"type":"v1","addr":"172.21.15.40:6807","nonce":22823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":22823},{"type":"v1","addr":"172.21.15.40:6805","nonce":22823}]},"public_addr":"172.21.15.40:6801/22823","cluster_addr":"172.21.15.40:6803/22823","heartbeat_back_addr":"172.21.15.40:6807/22823","heartbeat_front_addr":"172.21.15.40:6805/22823","state":["exists","up"]},{"osd":2,"uuid":"4d86bca7-743e-4983-b35f-36676c405b36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":22824},{"type":"v1","addr":"172.21.15.40:6809","nonce":22824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":22824},{"type":"v1","addr":"172.21.15.40:6811","nonce":22824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":22824},{"type":"v1","addr":"172.21.15.40:6815","nonce":22824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":22824},{"type":"v1","addr":"172.21.15.40:6813","nonce":22824}]},"public_addr":"172.21.15.40:6809/22824","cluster_addr":"172.21.15.40:6811/22824","heartbeat_back_addr":"172.21.15.40:6815/22824","heartbeat_front_addr":"172.21.15.40:6813/22824","state":["exists","up"]},{"osd":3,"uuid":"f4dc726b-6b34-4e8b-8eeb-87060b58002d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":22826},{"type":"v1","addr":"172.21.15.40:6819","nonce":22826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":22826},{"type":"v1","addr":"172.21.15.40:6823","nonce":22826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":22826},{"type":"v1","addr":"172.21.15.40:6831","nonce":22826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":22826},{"type":"v1","addr":"172.21.15.40:6827","nonce":22826}]},"public_addr":"172.21.15.40:6819/22826","cluster_addr":"172.21.15.40:6823/22826","heartbeat_back_addr":"172.21.15.40:6831/22826","heartbeat_front_addr":"172.21.15.40:6827/22826","state":["exists","up"]},{"osd":4,"uuid":"25a6d53b-d1f0-4498-b7d5-ffa38661d403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":23181},{"type":"v1","addr":"172.21.15.116:6819","nonce":23181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":23181},{"type":"v1","addr":"172.21.15.116:6821","nonce":23181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":23181},{"type":"v1","addr":"172.21.15.116:6825","nonce":23181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":23181},{"type":"v1","addr":"172.21.15.116:6823","nonce":23181}]},"public_addr":"172.21.15.116:6819/23181","cluster_addr":"172.21.15.116:6821/23181","heartbeat_back_addr":"172.21.15.116:6825/23181","heartbeat_front_addr":"172.21.15.116:6823/23181","state":["exists","up"]},{"osd":5,"uuid":"7d7a5af8-229b-4351-9a21-a3d509cd4635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":23171},{"type":"v1","addr":"172.21.15.116:6803","nonce":23171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":23171},{"type":"v1","addr":"172.21.15.116:6805","nonce":23171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":23171},{"type":"v1","addr":"172.21.15.116:6809","nonce":23171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":23171},{"type":"v1","addr":"172.21.15.116:6807","nonce":23171}]},"public_addr":"172.21.15.116:6803/23171","cluster_addr":"172.21.15.116:6805/23171","heartbeat_back_addr":"172.21.15.116:6809/23171","heartbeat_front_addr":"172.21.15.116:6807/23171","state":["exists","up"]},{"osd":6,"uuid":"8247d248-a140-4e36-9796-9847a97df58d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":23172},{"type":"v1","addr":"172.21.15.116:6811","nonce":23172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":23172},{"type":"v1","addr":"172.21.15.116:6813","nonce":23172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":23172},{"type":"v1","addr":"172.21.15.116:6817","nonce":23172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":23172},{"type":"v1","addr":"172.21.15.116:6815","nonce":23172}]},"public_addr":"172.21.15.116:6811/23172","cluster_addr":"172.21.15.116:6813/23172","heartbeat_back_addr":"172.21.15.116:6817/23172","heartbeat_front_addr":"172.21.15.116:6815/23172","state":["exists","up"]},{"osd":7,"uuid":"472803c4-70b8-499d-b18a-71aaeec6aab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":23196},{"type":"v1","addr":"172.21.15.116:6827","nonce":23196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":23196},{"type":"v1","addr":"172.21.15.116:6829","nonce":23196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":23196},{"type":"v1","addr":"172.21.15.116:6833","nonce":23196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":23196},{"type":"v1","addr":"172.21.15.116:6831","nonce":23196}]},"public_addr":"172.21.15.116:6827/23196","cluster_addr":"172.21.15.116:6829/23196","heartbeat_back_addr":"172.21.15.116:6833/23196","heartbeat_front_addr":"172.21.15.116:6831/23196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.029357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:11.989797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.091097+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.014766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.070860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.071864+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.063400+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.109028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-20T13:56:44.634 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.632+0000 7fafc63a3700 1 -- 172.21.15.40:0/1704568923 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fafa0051210 msgr2=0x7fafa00536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:44.634 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.632+0000 7fafc63a3700 1 --2- 172.21.15.40:0/1704568923 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fafa0051210 0x7fafa00536d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7fafb000b000 tx=0x7fafb0006dc0).stop 2020-11-20T13:56:44.634 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.632+0000 7fafc63a3700 1 -- 172.21.15.40:0/1704568923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafc00ae810 msgr2=0x7fafc00b8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:44.634 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.632+0000 7fafc63a3700 1 --2- 172.21.15.40:0/1704568923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafc00ae810 0x7fafc00b8da0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7fafb400c000 tx=0x7fafb4006dc0).stop 2020-11-20T13:56:44.635 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.632+0000 7fafc63a3700 1 -- 172.21.15.40:0/1704568923 shutdown_connections 2020-11-20T13:56:44.635 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.632+0000 7fafc63a3700 1 --2- 172.21.15.40:0/1704568923 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fafa0051210 0x7fafa00536d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:44.635 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.632+0000 7fafc63a3700 1 --2- 172.21.15.40:0/1704568923 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fafc0098f40 0x7fafc0146c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:44.635 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.632+0000 7fafc63a3700 1 --2- 172.21.15.40:0/1704568923 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fafc00b4b90 0x7fafc01404e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:44.635 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.632+0000 7fafc63a3700 1 --2- 172.21.15.40:0/1704568923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafc00ae810 0x7fafc00b8da0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:44.636 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.632+0000 7fafc63a3700 1 -- 172.21.15.40:0/1704568923 >> 172.21.15.40:0/1704568923 conn(0x7fafc00ba380 msgr2=0x7fafc0062990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:44.636 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.632+0000 7fafc63a3700 1 -- 172.21.15.40:0/1704568923 shutdown_connections 2020-11-20T13:56:44.636 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.633+0000 7fafc63a3700 1 -- 172.21.15.40:0/1704568923 wait complete. 2020-11-20T13:56:44.644 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T13:56:08.330017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T13:56:17.939684+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T13:56:20.563740+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T13:56:23.062623+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T13:56:44.645 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-20T13:56:44.762 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.761+0000 7f3074791700 1 Processor -- start 2020-11-20T13:56:44.762 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.761+0000 7f3074791700 1 -- start start 2020-11-20T13:56:44.764 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.762+0000 7f3074791700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f306c0b31f0 0x7f306c144bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:44.765 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.762+0000 7f3074791700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f306c0b1450 0x7f306c0b1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:44.765 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.762+0000 7f3074791700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f306c0b1df0 con 0x7f306c0b6c80 2020-11-20T13:56:44.765 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.762+0000 7f3074791700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f306c0b0d10 con 0x7f306c0b1450 2020-11-20T13:56:44.765 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.762+0000 7f3074791700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f306c0b0fb0 con 0x7f306c0b31f0 2020-11-20T13:56:44.766 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.762+0000 7f3072d2e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f306c0b1450 0x7f306c0b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:44.766 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.762+0000 7f307252d700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f306c0b6c80 0x7f306c0b40f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45878/0 (socket says 172.21.15.40:45878) 2020-11-20T13:56:44.766 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.762+0000 7f307252d700 1 -- 172.21.15.40:0/3997609861 learned_addr learned my addr 172.21.15.40:0/3997609861 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:44.766 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.762+0000 7f3071d2c700 1 --2- 172.21.15.40:0/3997609861 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f306c0b31f0 0x7f306c144bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:44.767 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.762+0000 7f3072d2e700 1 -- 172.21.15.40:0/3997609861 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f306c0b31f0 msgr2=0x7f306c144bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:44.768 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.762+0000 7f3072d2e700 1 --2- 172.21.15.40:0/3997609861 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f306c0b31f0 0x7f306c144bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:44.768 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.762+0000 7f3072d2e700 1 -- 172.21.15.40:0/3997609861 >> v1:172.21.15.40:6789/0 conn(0x7f306c0b6c80 legacy=0x7f306c0b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:44.768 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.762+0000 7f307152b700 1 -- 172.21.15.40:0/3997609861 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2882826234 0 0) 0x7f306c0b1df0 con 0x7f306c0b6c80 2020-11-20T13:56:44.768 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.762+0000 7f3072d2e700 1 -- 172.21.15.40:0/3997609861 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f306c0b7060 con 0x7f306c0b1450 2020-11-20T13:56:44.768 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.762+0000 7f3071d2c700 1 --2- 172.21.15.40:0/3997609861 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f306c0b31f0 0x7f306c144bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:44.769 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.762+0000 7f3072d2e700 1 --2- 172.21.15.40:0/3997609861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f306c0b1450 0x7f306c0b1900 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f306c0b3100 tx=0x7f305c006dc0).ready entity=mon.0 client_cookie=4686e04ce7c58c33 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:44.769 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.762+0000 7f307152b700 1 -- 172.21.15.40:0/3997609861 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f305c0097b0 con 0x7f306c0b1450 2020-11-20T13:56:44.769 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.762+0000 7f307152b700 1 -- 172.21.15.40:0/3997609861 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f305c009910 con 0x7f306c0b1450 2020-11-20T13:56:44.769 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.762+0000 7f307152b700 1 -- 172.21.15.40:0/3997609861 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f305c009c70 con 0x7f306c0b1450 2020-11-20T13:56:44.769 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.762+0000 7f3074791700 1 -- 172.21.15.40:0/3997609861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f306c0b1450 msgr2=0x7f306c0b1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:44.770 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.762+0000 7f3074791700 1 --2- 172.21.15.40:0/3997609861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f306c0b1450 0x7f306c0b1900 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f306c0b3100 tx=0x7f305c006dc0).stop 2020-11-20T13:56:44.770 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.763+0000 7f3074791700 1 -- 172.21.15.40:0/3997609861 shutdown_connections 2020-11-20T13:56:44.770 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.763+0000 7f3074791700 1 --2- 172.21.15.40:0/3997609861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f306c0b1450 0x7f306c0b1900 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:44.770 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.763+0000 7f3074791700 1 --2- 172.21.15.40:0/3997609861 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f306c0b31f0 0x7f306c144bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:44.771 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.763+0000 7f3074791700 1 -- 172.21.15.40:0/3997609861 >> 172.21.15.40:0/3997609861 conn(0x7f306c0ba1d0 msgr2=0x7f306c0b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:44.771 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.763+0000 7f3074791700 1 -- 172.21.15.40:0/3997609861 shutdown_connections 2020-11-20T13:56:44.771 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.763+0000 7f3074791700 1 -- 172.21.15.40:0/3997609861 wait complete. 2020-11-20T13:56:44.774 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.764+0000 7f3074791700 1 Processor -- start 2020-11-20T13:56:44.774 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.764+0000 7f3074791700 1 -- start start 2020-11-20T13:56:44.774 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.765+0000 7f3074791700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f306c0b1450 0x7f306c12aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:44.774 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.765+0000 7f3074791700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f306c0b31f0 0x7f306c0af6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:44.774 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.765+0000 7f3074791700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f306c0b6c80 0x7f306c05f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:44.775 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.765+0000 7f3074791700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306c1565a0 con 0x7f306c0b31f0 2020-11-20T13:56:44.775 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.765+0000 7f3074791700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306c1567b0 con 0x7f306c0b6c80 2020-11-20T13:56:44.775 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.765+0000 7f3074791700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f306c156a50 con 0x7f306c0b1450 2020-11-20T13:56:44.775 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.765+0000 7f3071d2c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f306c0b31f0 0x7f306c0af6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:44.775 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.765+0000 7f3071d2c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f306c0b31f0 0x7f306c0af6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55288/0 (socket says 172.21.15.40:55288) 2020-11-20T13:56:44.776 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.765+0000 7f3071d2c700 1 -- 172.21.15.40:0/3740293184 learned_addr learned my addr 172.21.15.40:0/3740293184 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:44.776 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.765+0000 7f307252d700 1 --2- 172.21.15.40:0/3740293184 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f306c0b1450 0x7f306c12aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:44.776 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.765+0000 7f3072d2e700 1 --2- 172.21.15.40:0/3740293184 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f306c0b6c80 0x7f306c05f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:44.776 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.765+0000 7f3071d2c700 1 -- 172.21.15.40:0/3740293184 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f306c0b1450 msgr2=0x7f306c12aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:44.776 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.765+0000 7f3071d2c700 1 --2- 172.21.15.40:0/3740293184 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f306c0b1450 0x7f306c12aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:44.777 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.765+0000 7f3071d2c700 1 -- 172.21.15.40:0/3740293184 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f306c0b6c80 msgr2=0x7f306c05f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:44.777 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.765+0000 7f3071d2c700 1 --2- 172.21.15.40:0/3740293184 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f306c0b6c80 0x7f306c05f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:44.777 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.765+0000 7f3071d2c700 1 -- 172.21.15.40:0/3740293184 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f306c12af60 con 0x7f306c0b31f0 2020-11-20T13:56:44.778 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.765+0000 7f307252d700 1 --2- 172.21.15.40:0/3740293184 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f306c0b1450 0x7f306c12aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:44.778 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.765+0000 7f3071d2c700 1 --2- 172.21.15.40:0/3740293184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f306c0b31f0 0x7f306c0af6b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f3060007000 tx=0x7f3060008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:44.778 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.765+0000 7f306b7fe700 1 -- 172.21.15.40:0/3740293184 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3060007750 con 0x7f306c0b31f0 2020-11-20T13:56:44.778 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.765+0000 7f306b7fe700 1 -- 172.21.15.40:0/3740293184 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30600078b0 con 0x7f306c0b31f0 2020-11-20T13:56:44.778 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.765+0000 7f3074791700 1 -- 172.21.15.40:0/3740293184 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f306c156ef0 con 0x7f306c0b31f0 2020-11-20T13:56:44.779 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.765+0000 7f306b7fe700 1 -- 172.21.15.40:0/3740293184 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3060007c10 con 0x7f306c0b31f0 2020-11-20T13:56:44.779 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.766+0000 7f3074791700 1 -- 172.21.15.40:0/3740293184 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f306c131280 con 0x7f306c0b31f0 2020-11-20T13:56:44.779 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.767+0000 7f306b7fe700 1 -- 172.21.15.40:0/3740293184 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f3060007d70 con 0x7f306c0b31f0 2020-11-20T13:56:44.779 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.767+0000 7f3074791700 1 -- 172.21.15.40:0/3740293184 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f306c1570e0 con 0x7f306c0b31f0 2020-11-20T13:56:44.779 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.767+0000 7f306b7fe700 1 --2- 172.21.15.40:0/3740293184 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f304c0513c0 0x7f304c053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:44.780 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.767+0000 7f306b7fe700 1 -- 172.21.15.40:0/3740293184 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f3060069390 con 0x7f306c0b31f0 2020-11-20T13:56:44.780 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.767+0000 7f307252d700 1 --2- 172.21.15.40:0/3740293184 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f304c0513c0 0x7f304c053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:44.780 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.767+0000 7f307252d700 1 --2- 172.21.15.40:0/3740293184 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f304c0513c0 0x7f304c053880 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f306400b000 tx=0x7f3064009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:44.780 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.770+0000 7f306b7fe700 1 -- 172.21.15.40:0/3740293184 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f30600415b0 con 0x7f306c0b31f0 2020-11-20T13:56:44.931 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.930+0000 7f3074791700 1 -- 172.21.15.40:0/3740293184 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f306c156330 con 0x7f306c0b31f0 2020-11-20T13:56:44.932 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.930+0000 7f306b7fe700 1 -- 172.21.15.40:0/3740293184 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v24) v1 ==== 110+0+10 (secure 0 0 0) 0x7f3060044bd0 con 0x7f306c0b31f0 2020-11-20T13:56:44.932 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2020-11-20T13:56:44.934 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.932+0000 7f3074791700 1 -- 172.21.15.40:0/3740293184 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f304c0513c0 msgr2=0x7f304c053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:44.934 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.932+0000 7f3074791700 1 --2- 172.21.15.40:0/3740293184 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f304c0513c0 0x7f304c053880 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f306400b000 tx=0x7f3064009040).stop 2020-11-20T13:56:44.934 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.932+0000 7f3074791700 1 -- 172.21.15.40:0/3740293184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f306c0b31f0 msgr2=0x7f306c0af6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:44.934 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.932+0000 7f3074791700 1 --2- 172.21.15.40:0/3740293184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f306c0b31f0 0x7f306c0af6b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f3060007000 tx=0x7f3060008040).stop 2020-11-20T13:56:44.935 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.932+0000 7f3074791700 1 -- 172.21.15.40:0/3740293184 shutdown_connections 2020-11-20T13:56:44.935 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.932+0000 7f3074791700 1 --2- 172.21.15.40:0/3740293184 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f304c0513c0 0x7f304c053880 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:44.936 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.932+0000 7f3074791700 1 --2- 172.21.15.40:0/3740293184 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f306c0b1450 0x7f306c12aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:44.936 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.932+0000 7f3074791700 1 --2- 172.21.15.40:0/3740293184 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f306c0b6c80 0x7f306c05f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:44.936 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.932+0000 7f3074791700 1 --2- 172.21.15.40:0/3740293184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f306c0b31f0 0x7f306c0af6b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:44.937 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.932+0000 7f3074791700 1 -- 172.21.15.40:0/3740293184 >> 172.21.15.40:0/3740293184 conn(0x7f306c0ba1d0 msgr2=0x7f306c0b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:44.937 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.932+0000 7f3074791700 1 -- 172.21.15.40:0/3740293184 shutdown_connections 2020-11-20T13:56:44.937 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:44.932+0000 7f3074791700 1 -- 172.21.15.40:0/3740293184 wait complete. 2020-11-20T13:56:44.943 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-20T13:56:45.060 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.059+0000 7f5ca2ab9700 1 Processor -- start 2020-11-20T13:56:45.060 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.059+0000 7f5ca2ab9700 1 -- start start 2020-11-20T13:56:45.062 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.060+0000 7f5ca2ab9700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5c9c0b6c80 0x7f5c9c0b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:45.063 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.060+0000 7f5ca2ab9700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5c9c144bd0 0x7f5c9c0b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:45.063 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.060+0000 7f5ca2ab9700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c9c0b0d10 con 0x7f5c9c0b45e0 2020-11-20T13:56:45.063 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.060+0000 7f5ca2ab9700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c9c145030 con 0x7f5c9c0b6c80 2020-11-20T13:56:45.063 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.060+0000 7f5ca2ab9700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c0b0fb0 con 0x7f5c9c144bd0 2020-11-20T13:56:45.063 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.060+0000 7f5ca0855700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5c9c0b6c80 0x7f5c9c0b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:45.064 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.060+0000 7f5ca0855700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5c9c0b6c80 0x7f5c9c0b31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55294/0 (socket says 172.21.15.40:55294) 2020-11-20T13:56:45.064 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.060+0000 7f5ca0855700 1 -- 172.21.15.40:0/2060570633 learned_addr learned my addr 172.21.15.40:0/2060570633 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:45.064 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.060+0000 7f5c9bfff700 1 --2- 172.21.15.40:0/2060570633 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5c9c144bd0 0x7f5c9c0b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:45.064 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.060+0000 7f5ca0855700 1 -- 172.21.15.40:0/2060570633 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5c9c144bd0 msgr2=0x7f5c9c0b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:45.065 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.060+0000 7f5ca0855700 1 --2- 172.21.15.40:0/2060570633 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5c9c144bd0 0x7f5c9c0b40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:45.065 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.060+0000 7f5ca0855700 1 -- 172.21.15.40:0/2060570633 >> v1:172.21.15.116:6789/0 conn(0x7f5c9c0b45e0 legacy=0x7f5c9c0b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:45.065 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.060+0000 7f5ca0855700 1 -- 172.21.15.40:0/2060570633 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c9c0b7060 con 0x7f5c9c0b6c80 2020-11-20T13:56:45.065 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.060+0000 7f5ca0855700 1 --2- 172.21.15.40:0/2060570633 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c9c0b6c80 0x7f5c9c0b31f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f5c90002f30 tx=0x7f5c90006e20).ready entity=mon.0 client_cookie=cdb45e97428e0023 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:45.065 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.060+0000 7f5c9b7fe700 1 -- 172.21.15.40:0/2060570633 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c900097f0 con 0x7f5c9c0b6c80 2020-11-20T13:56:45.066 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.060+0000 7f5c9b7fe700 1 -- 172.21.15.40:0/2060570633 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c90009950 con 0x7f5c9c0b6c80 2020-11-20T13:56:45.066 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.060+0000 7f5c9b7fe700 1 -- 172.21.15.40:0/2060570633 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c90009cb0 con 0x7f5c9c0b6c80 2020-11-20T13:56:45.066 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.060+0000 7f5ca2ab9700 1 -- 172.21.15.40:0/2060570633 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c9c0b6c80 msgr2=0x7f5c9c0b31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:45.066 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.060+0000 7f5ca2ab9700 1 --2- 172.21.15.40:0/2060570633 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c9c0b6c80 0x7f5c9c0b31f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f5c90002f30 tx=0x7f5c90006e20).stop 2020-11-20T13:56:45.066 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.061+0000 7f5ca2ab9700 1 -- 172.21.15.40:0/2060570633 shutdown_connections 2020-11-20T13:56:45.067 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.061+0000 7f5ca2ab9700 1 --2- 172.21.15.40:0/2060570633 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c9c0b6c80 0x7f5c9c0b31f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:45.067 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.061+0000 7f5ca2ab9700 1 --2- 172.21.15.40:0/2060570633 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5c9c144bd0 0x7f5c9c0b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:45.067 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.061+0000 7f5ca2ab9700 1 -- 172.21.15.40:0/2060570633 >> 172.21.15.40:0/2060570633 conn(0x7f5c9c0ba1d0 msgr2=0x7f5c9c0b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:45.067 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.061+0000 7f5ca2ab9700 1 -- 172.21.15.40:0/2060570633 shutdown_connections 2020-11-20T13:56:45.068 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.061+0000 7f5ca2ab9700 1 -- 172.21.15.40:0/2060570633 wait complete. 2020-11-20T13:56:45.070 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.062+0000 7f5ca2ab9700 1 Processor -- start 2020-11-20T13:56:45.071 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.062+0000 7f5ca2ab9700 1 -- start start 2020-11-20T13:56:45.071 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.063+0000 7f5ca2ab9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c9c0b45e0 0x7f5c9c067840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:45.071 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.063+0000 7f5ca2ab9700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5c9c0b6c80 0x7f5c9c1478a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:45.071 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.063+0000 7f5ca2ab9700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c9c144bd0 0x7f5c9c157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:45.071 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.063+0000 7f5ca2ab9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c156950 con 0x7f5c9c0b45e0 2020-11-20T13:56:45.072 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.063+0000 7f5ca2ab9700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c156bf0 con 0x7f5c9c144bd0 2020-11-20T13:56:45.072 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.063+0000 7f5ca2ab9700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c156680 con 0x7f5c9c0b6c80 2020-11-20T13:56:45.072 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.063+0000 7f5ca0855700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c9c0b45e0 0x7f5c9c067840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:45.072 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.063+0000 7f5ca0855700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c9c0b45e0 0x7f5c9c067840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55300/0 (socket says 172.21.15.40:55300) 2020-11-20T13:56:45.073 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.063+0000 7f5ca0855700 1 -- 172.21.15.40:0/3167783326 learned_addr learned my addr 172.21.15.40:0/3167783326 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:45.073 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.063+0000 7f5c9bfff700 1 --2- 172.21.15.40:0/3167783326 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5c9c0b6c80 0x7f5c9c1478a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:45.073 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.063+0000 7f5ca1056700 1 --2- 172.21.15.40:0/3167783326 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c9c144bd0 0x7f5c9c157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:45.073 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.063+0000 7f5ca0855700 1 -- 172.21.15.40:0/3167783326 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5c9c0b6c80 msgr2=0x7f5c9c1478a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:45.073 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.063+0000 7f5ca0855700 1 --2- 172.21.15.40:0/3167783326 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5c9c0b6c80 0x7f5c9c1478a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:45.074 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.063+0000 7f5ca0855700 1 -- 172.21.15.40:0/3167783326 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c9c144bd0 msgr2=0x7f5c9c157090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:45.074 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.063+0000 7f5ca0855700 1 --2- 172.21.15.40:0/3167783326 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c9c144bd0 0x7f5c9c157090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:45.074 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.063+0000 7f5ca0855700 1 -- 172.21.15.40:0/3167783326 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c9c157580 con 0x7f5c9c0b45e0 2020-11-20T13:56:45.074 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.063+0000 7f5ca1056700 1 --2- 172.21.15.40:0/3167783326 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c9c144bd0 0x7f5c9c157090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:45.075 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.063+0000 7f5ca0855700 1 --2- 172.21.15.40:0/3167783326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c9c0b45e0 0x7f5c9c067840 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f5c90002f30 tx=0x7f5c90015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:45.075 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.063+0000 7f5c99ffb700 1 -- 172.21.15.40:0/3167783326 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c90021070 con 0x7f5c9c0b45e0 2020-11-20T13:56:45.075 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.063+0000 7f5c99ffb700 1 -- 172.21.15.40:0/3167783326 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c90011ce0 con 0x7f5c9c0b45e0 2020-11-20T13:56:45.075 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.063+0000 7f5ca2ab9700 1 -- 172.21.15.40:0/3167783326 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c9c063c10 con 0x7f5c9c0b45e0 2020-11-20T13:56:45.075 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.063+0000 7f5c99ffb700 1 -- 172.21.15.40:0/3167783326 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c90006210 con 0x7f5c9c0b45e0 2020-11-20T13:56:45.076 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.063+0000 7f5ca2ab9700 1 -- 172.21.15.40:0/3167783326 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c9c112870 con 0x7f5c9c0b45e0 2020-11-20T13:56:45.076 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.064+0000 7f5ca2ab9700 1 -- 172.21.15.40:0/3167783326 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c9c156db0 con 0x7f5c9c0b45e0 2020-11-20T13:56:45.076 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.065+0000 7f5c99ffb700 1 -- 172.21.15.40:0/3167783326 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f5c9000f070 con 0x7f5c9c0b45e0 2020-11-20T13:56:45.076 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.065+0000 7f5c99ffb700 1 --2- 172.21.15.40:0/3167783326 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f5c84051320 0x7f5c840537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:45.076 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.065+0000 7f5c99ffb700 1 -- 172.21.15.40:0/3167783326 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f5c9006a7b0 con 0x7f5c9c0b45e0 2020-11-20T13:56:45.077 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.065+0000 7f5c9bfff700 1 --2- 172.21.15.40:0/3167783326 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f5c84051320 0x7f5c840537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:45.077 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.065+0000 7f5c9bfff700 1 --2- 172.21.15.40:0/3167783326 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f5c84051320 0x7f5c840537e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f5c8800b000 tx=0x7f5c88006dc0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:45.077 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.068+0000 7f5c99ffb700 1 -- 172.21.15.40:0/3167783326 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f5c900429d0 con 0x7f5c9c0b45e0 2020-11-20T13:56:45.230 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.228+0000 7f5ca2ab9700 1 -- 172.21.15.40:0/3167783326 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5c9c057050 con 0x7f5c9c0b45e0 2020-11-20T13:56:45.230 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.229+0000 7f5c99ffb700 1 -- 172.21.15.40:0/3167783326 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5c90045ff0 con 0x7f5c9c0b45e0 2020-11-20T13:56:45.230 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2020-11-20T13:56:45.231 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.230+0000 7f5ca2ab9700 1 -- 172.21.15.40:0/3167783326 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f5c84051320 msgr2=0x7f5c840537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:45.232 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.230+0000 7f5ca2ab9700 1 --2- 172.21.15.40:0/3167783326 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f5c84051320 0x7f5c840537e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f5c8800b000 tx=0x7f5c88006dc0).stop 2020-11-20T13:56:45.232 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.230+0000 7f5ca2ab9700 1 -- 172.21.15.40:0/3167783326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c9c0b45e0 msgr2=0x7f5c9c067840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:45.232 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.230+0000 7f5ca2ab9700 1 --2- 172.21.15.40:0/3167783326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c9c0b45e0 0x7f5c9c067840 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f5c90002f30 tx=0x7f5c90015000).stop 2020-11-20T13:56:45.232 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.230+0000 7f5ca2ab9700 1 -- 172.21.15.40:0/3167783326 shutdown_connections 2020-11-20T13:56:45.233 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.230+0000 7f5ca2ab9700 1 --2- 172.21.15.40:0/3167783326 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f5c84051320 0x7f5c840537e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:45.233 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.230+0000 7f5ca2ab9700 1 --2- 172.21.15.40:0/3167783326 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c9c144bd0 0x7f5c9c157090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:45.233 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.230+0000 7f5ca2ab9700 1 --2- 172.21.15.40:0/3167783326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c9c0b45e0 0x7f5c9c067840 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:45.233 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.230+0000 7f5ca2ab9700 1 --2- 172.21.15.40:0/3167783326 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5c9c0b6c80 0x7f5c9c1478a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:45.233 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.230+0000 7f5ca2ab9700 1 -- 172.21.15.40:0/3167783326 >> 172.21.15.40:0/3167783326 conn(0x7f5c9c0ba1d0 msgr2=0x7f5c9c0b5210 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:45.234 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.230+0000 7f5ca2ab9700 1 -- 172.21.15.40:0/3167783326 shutdown_connections 2020-11-20T13:56:45.234 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.230+0000 7f5ca2ab9700 1 -- 172.21.15.40:0/3167783326 wait complete. 2020-11-20T13:56:45.242 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-20T13:56:45.360 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.359+0000 7fa055047700 1 Processor -- start 2020-11-20T13:56:45.360 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.359+0000 7fa055047700 1 -- start start 2020-11-20T13:56:45.363 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.359+0000 7fa055047700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa0500b9270 0x7fa0500b82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:45.363 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.359+0000 7fa055047700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa05005c190 0x7fa0500b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:45.363 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.359+0000 7fa055047700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa050144d30 con 0x7fa0500b4460 2020-11-20T13:56:45.363 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.359+0000 7fa055047700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0500b9650 con 0x7fa0500b9270 2020-11-20T13:56:45.363 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.359+0000 7fa055047700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa05005c6a0 con 0x7fa05005c190 2020-11-20T13:56:45.364 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.360+0000 7fa04f7fe700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa05005c190 0x7fa0500b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:45.364 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.360+0000 7fa04ffff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa0500b9270 0x7fa0500b82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:45.364 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.360+0000 7fa04f7fe700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa05005c190 0x7fa0500b3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.40:40384/0 (socket says 172.21.15.40:40384) 2020-11-20T13:56:45.364 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.360+0000 7fa054846700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fa0500b4460 0x7fa050147710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.40:58540/0 (socket says 172.21.15.40:58540) 2020-11-20T13:56:45.364 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.360+0000 7fa04f7fe700 1 -- 172.21.15.40:0/956285927 learned_addr learned my addr 172.21.15.40:0/956285927 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:45.365 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.360+0000 7fa04f7fe700 1 -- 172.21.15.40:0/956285927 >> v2:172.21.15.116:3300/0 conn(0x7fa0500b9270 msgr2=0x7fa0500b82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:45.365 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.360+0000 7fa04f7fe700 1 --2- 172.21.15.40:0/956285927 >> v2:172.21.15.116:3300/0 conn(0x7fa0500b9270 0x7fa0500b82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:45.365 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.360+0000 7fa04f7fe700 1 -- 172.21.15.40:0/956285927 >> v1:172.21.15.116:6789/0 conn(0x7fa0500b4460 legacy=0x7fa050147710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:45.365 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.360+0000 7fa04f7fe700 1 -- 172.21.15.40:0/956285927 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa05005c940 con 0x7fa05005c190 2020-11-20T13:56:45.366 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.360+0000 7fa04f7fe700 1 --2- 172.21.15.40:0/956285927 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa05005c190 0x7fa0500b3f70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fa044002f30 tx=0x7fa044006e80).ready entity=mon.2 client_cookie=5562509cfdad0290 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:45.366 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.360+0000 7fa04effd700 1 -- 172.21.15.40:0/956285927 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0440097f0 con 0x7fa05005c190 2020-11-20T13:56:45.366 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.360+0000 7fa04effd700 1 -- 172.21.15.40:0/956285927 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa044009950 con 0x7fa05005c190 2020-11-20T13:56:45.366 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.360+0000 7fa04effd700 1 -- 172.21.15.40:0/956285927 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa044009cb0 con 0x7fa05005c190 2020-11-20T13:56:45.366 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.360+0000 7fa055047700 1 -- 172.21.15.40:0/956285927 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa05005c190 msgr2=0x7fa0500b3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:45.367 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.360+0000 7fa055047700 1 --2- 172.21.15.40:0/956285927 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa05005c190 0x7fa0500b3f70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fa044002f30 tx=0x7fa044006e80).stop 2020-11-20T13:56:45.367 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.361+0000 7fa055047700 1 -- 172.21.15.40:0/956285927 shutdown_connections 2020-11-20T13:56:45.367 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.361+0000 7fa055047700 1 --2- 172.21.15.40:0/956285927 >> v2:172.21.15.116:3300/0 conn(0x7fa0500b9270 0x7fa0500b82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:45.367 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.361+0000 7fa055047700 1 --2- 172.21.15.40:0/956285927 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa05005c190 0x7fa0500b3f70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:45.367 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.361+0000 7fa055047700 1 -- 172.21.15.40:0/956285927 >> 172.21.15.40:0/956285927 conn(0x7fa0500bd940 msgr2=0x7fa0500bdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:45.368 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.361+0000 7fa055047700 1 -- 172.21.15.40:0/956285927 shutdown_connections 2020-11-20T13:56:45.368 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.361+0000 7fa055047700 1 -- 172.21.15.40:0/956285927 wait complete. 2020-11-20T13:56:45.368 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.362+0000 7fa055047700 1 Processor -- start 2020-11-20T13:56:45.368 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.362+0000 7fa055047700 1 -- start start 2020-11-20T13:56:45.371 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.363+0000 7fa055047700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa05005c190 0x7fa0500b3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:45.371 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.363+0000 7fa055047700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa0500b4460 0x7fa05014b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:45.371 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.363+0000 7fa055047700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0500b9270 0x7fa050141bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:45.371 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.363+0000 7fa055047700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0500accf0 con 0x7fa0500b9270 2020-11-20T13:56:45.371 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.363+0000 7fa055047700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0500b3700 con 0x7fa05005c190 2020-11-20T13:56:45.372 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.363+0000 7fa055047700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0500b3990 con 0x7fa0500b4460 2020-11-20T13:56:45.372 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.363+0000 7fa054846700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0500b9270 0x7fa050141bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:45.372 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.363+0000 7fa04ffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa05005c190 0x7fa0500b3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:45.372 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.363+0000 7fa054846700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0500b9270 0x7fa050141bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55318/0 (socket says 172.21.15.40:55318) 2020-11-20T13:56:45.373 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.363+0000 7fa054846700 1 -- 172.21.15.40:0/1009747629 learned_addr learned my addr 172.21.15.40:0/1009747629 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:45.373 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.363+0000 7fa04f7fe700 1 --2- 172.21.15.40:0/1009747629 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa0500b4460 0x7fa05014b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:45.373 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.363+0000 7fa054846700 1 -- 172.21.15.40:0/1009747629 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa0500b4460 msgr2=0x7fa05014b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:45.373 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.363+0000 7fa054846700 1 --2- 172.21.15.40:0/1009747629 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa0500b4460 0x7fa05014b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:45.373 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.363+0000 7fa054846700 1 -- 172.21.15.40:0/1009747629 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa05005c190 msgr2=0x7fa0500b3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:45.374 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.363+0000 7fa054846700 1 --2- 172.21.15.40:0/1009747629 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa05005c190 0x7fa0500b3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:45.374 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.363+0000 7fa054846700 1 -- 172.21.15.40:0/1009747629 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0500b3c30 con 0x7fa0500b9270 2020-11-20T13:56:45.374 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.363+0000 7fa054846700 1 --2- 172.21.15.40:0/1009747629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0500b9270 0x7fa050141bd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7fa05005cc60 tx=0x7fa048007e80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:45.374 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.363+0000 7fa04d7fa700 1 -- 172.21.15.40:0/1009747629 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa048005ab0 con 0x7fa0500b9270 2020-11-20T13:56:45.374 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.363+0000 7fa04d7fa700 1 -- 172.21.15.40:0/1009747629 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa048005c10 con 0x7fa0500b9270 2020-11-20T13:56:45.375 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.363+0000 7fa055047700 1 -- 172.21.15.40:0/1009747629 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa05014bed0 con 0x7fa0500b9270 2020-11-20T13:56:45.375 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.363+0000 7fa04d7fa700 1 -- 172.21.15.40:0/1009747629 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa048007110 con 0x7fa0500b9270 2020-11-20T13:56:45.375 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.363+0000 7fa055047700 1 -- 172.21.15.40:0/1009747629 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0501570f0 con 0x7fa0500b9270 2020-11-20T13:56:45.375 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.364+0000 7fa055047700 1 -- 172.21.15.40:0/1009747629 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0500ace60 con 0x7fa0500b9270 2020-11-20T13:56:45.375 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.365+0000 7fa04d7fa700 1 -- 172.21.15.40:0/1009747629 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fa048010030 con 0x7fa0500b9270 2020-11-20T13:56:45.376 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.365+0000 7fa04d7fa700 1 --2- 172.21.15.40:0/1009747629 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa030051290 0x7fa030053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:45.376 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.365+0000 7fa04d7fa700 1 -- 172.21.15.40:0/1009747629 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fa048069110 con 0x7fa0500b9270 2020-11-20T13:56:45.376 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.365+0000 7fa04ffff700 1 --2- 172.21.15.40:0/1009747629 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa030051290 0x7fa030053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:45.376 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.365+0000 7fa04ffff700 1 --2- 172.21.15.40:0/1009747629 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa030051290 0x7fa030053750 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fa04000b000 tx=0x7fa040006dc0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:45.376 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.368+0000 7fa04d7fa700 1 -- 172.21.15.40:0/1009747629 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fa048041330 con 0x7fa0500b9270 2020-11-20T13:56:45.530 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.529+0000 7fa055047700 1 -- 172.21.15.40:0/1009747629 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa050057050 con 0x7fa0500b9270 2020-11-20T13:56:45.530 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.529+0000 7fa04d7fa700 1 -- 172.21.15.40:0/1009747629 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+10 (secure 0 0 0) 0x7fa048044950 con 0x7fa0500b9270 2020-11-20T13:56:45.531 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2020-11-20T13:56:45.532 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.531+0000 7fa055047700 1 -- 172.21.15.40:0/1009747629 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa030051290 msgr2=0x7fa030053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:45.532 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.531+0000 7fa055047700 1 --2- 172.21.15.40:0/1009747629 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa030051290 0x7fa030053750 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fa04000b000 tx=0x7fa040006dc0).stop 2020-11-20T13:56:45.533 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.531+0000 7fa055047700 1 -- 172.21.15.40:0/1009747629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0500b9270 msgr2=0x7fa050141bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:45.533 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.531+0000 7fa055047700 1 --2- 172.21.15.40:0/1009747629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0500b9270 0x7fa050141bd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7fa05005cc60 tx=0x7fa048007e80).stop 2020-11-20T13:56:45.533 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.531+0000 7fa055047700 1 -- 172.21.15.40:0/1009747629 shutdown_connections 2020-11-20T13:56:45.533 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.531+0000 7fa055047700 1 --2- 172.21.15.40:0/1009747629 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa030051290 0x7fa030053750 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:45.534 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.531+0000 7fa055047700 1 --2- 172.21.15.40:0/1009747629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0500b9270 0x7fa050141bd0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:45.534 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.531+0000 7fa055047700 1 --2- 172.21.15.40:0/1009747629 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa05005c190 0x7fa0500b3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:45.535 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.531+0000 7fa055047700 1 --2- 172.21.15.40:0/1009747629 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa0500b4460 0x7fa05014b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:45.535 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.531+0000 7fa055047700 1 -- 172.21.15.40:0/1009747629 >> 172.21.15.40:0/1009747629 conn(0x7fa0500bd940 msgr2=0x7fa050147c60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:45.535 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.531+0000 7fa055047700 1 -- 172.21.15.40:0/1009747629 shutdown_connections 2020-11-20T13:56:45.535 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.531+0000 7fa055047700 1 -- 172.21.15.40:0/1009747629 wait complete. 2020-11-20T13:56:45.542 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-20T13:56:45.659 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.657+0000 7f0cadb3a700 1 Processor -- start 2020-11-20T13:56:45.659 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.657+0000 7f0cadb3a700 1 -- start start 2020-11-20T13:56:45.661 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.658+0000 7f0cadb3a700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0ca80b5bf0 0x7f0ca805f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:45.661 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.658+0000 7f0cadb3a700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0ca8144390 0x7f0ca805f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:45.662 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.658+0000 7f0cadb3a700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0ca805fd70 0x7f0ca80b2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:45.662 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.658+0000 7f0cadb3a700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ca80b5fd0 con 0x7f0ca80b5bf0 2020-11-20T13:56:45.662 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.658+0000 7f0cadb3a700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ca80b05f0 con 0x7f0ca8144390 2020-11-20T13:56:45.662 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.658+0000 7f0cadb3a700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca8144870 con 0x7f0ca805fd70 2020-11-20T13:56:45.662 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.658+0000 7f0ca77fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0ca80b5bf0 0x7f0ca805f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:45.663 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.658+0000 7f0ca77fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0ca80b5bf0 0x7f0ca805f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55322/0 (socket says 172.21.15.40:55322) 2020-11-20T13:56:45.663 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.658+0000 7f0ca77fe700 1 -- 172.21.15.40:0/1670416550 learned_addr learned my addr 172.21.15.40:0/1670416550 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:45.663 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.658+0000 7f0ca7fff700 1 --2- 172.21.15.40:0/1670416550 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0ca805fd70 0x7f0ca80b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:45.663 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.659+0000 7f0ca6ffd700 1 --2- 172.21.15.40:0/1670416550 >> v2:172.21.15.116:3300/0 conn(0x7f0ca8144390 0x7f0ca805f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:45.663 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.659+0000 7f0ca77fe700 1 -- 172.21.15.40:0/1670416550 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0ca805fd70 msgr2=0x7f0ca80b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:45.664 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.659+0000 7f0ca77fe700 1 --2- 172.21.15.40:0/1670416550 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0ca805fd70 0x7f0ca80b2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:45.664 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.659+0000 7f0ca77fe700 1 -- 172.21.15.40:0/1670416550 >> v2:172.21.15.116:3300/0 conn(0x7f0ca8144390 msgr2=0x7f0ca805f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:45.664 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.659+0000 7f0ca77fe700 1 --2- 172.21.15.40:0/1670416550 >> v2:172.21.15.116:3300/0 conn(0x7f0ca8144390 0x7f0ca805f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:45.664 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.659+0000 7f0ca77fe700 1 -- 172.21.15.40:0/1670416550 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca80b0c30 con 0x7f0ca80b5bf0 2020-11-20T13:56:45.664 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.659+0000 7f0ca77fe700 1 --2- 172.21.15.40:0/1670416550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ca80b5bf0 0x7f0ca805f390 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f0c98002f30 tx=0x7f0c98006e20).ready entity=mon.0 client_cookie=8fc24423f7087d69 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:45.665 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.659+0000 7f0ca67fc700 1 -- 172.21.15.40:0/1670416550 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c98014070 con 0x7f0ca80b5bf0 2020-11-20T13:56:45.665 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.659+0000 7f0ca67fc700 1 -- 172.21.15.40:0/1670416550 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c98009ab0 con 0x7f0ca80b5bf0 2020-11-20T13:56:45.665 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.659+0000 7f0ca67fc700 1 -- 172.21.15.40:0/1670416550 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c98009e10 con 0x7f0ca80b5bf0 2020-11-20T13:56:45.665 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.659+0000 7f0cadb3a700 1 -- 172.21.15.40:0/1670416550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ca80b5bf0 msgr2=0x7f0ca805f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:45.666 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.659+0000 7f0cadb3a700 1 --2- 172.21.15.40:0/1670416550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ca80b5bf0 0x7f0ca805f390 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f0c98002f30 tx=0x7f0c98006e20).stop 2020-11-20T13:56:45.666 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.659+0000 7f0cadb3a700 1 -- 172.21.15.40:0/1670416550 shutdown_connections 2020-11-20T13:56:45.666 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.659+0000 7f0cadb3a700 1 --2- 172.21.15.40:0/1670416550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ca80b5bf0 0x7f0ca805f390 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:45.667 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.659+0000 7f0cadb3a700 1 --2- 172.21.15.40:0/1670416550 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0ca805fd70 0x7f0ca80b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:45.667 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.659+0000 7f0cadb3a700 1 --2- 172.21.15.40:0/1670416550 >> v2:172.21.15.116:3300/0 conn(0x7f0ca8144390 0x7f0ca805f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:45.667 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.659+0000 7f0cadb3a700 1 -- 172.21.15.40:0/1670416550 >> 172.21.15.40:0/1670416550 conn(0x7f0ca80ba1d0 msgr2=0x7f0ca80b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:45.668 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.660+0000 7f0cadb3a700 1 -- 172.21.15.40:0/1670416550 shutdown_connections 2020-11-20T13:56:45.668 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.660+0000 7f0cadb3a700 1 -- 172.21.15.40:0/1670416550 wait complete. 2020-11-20T13:56:45.668 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.661+0000 7f0cadb3a700 1 Processor -- start 2020-11-20T13:56:45.671 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.661+0000 7f0cadb3a700 1 -- start start 2020-11-20T13:56:45.671 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.661+0000 7f0cadb3a700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0ca805fd70 0x7f0ca813ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:45.671 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.661+0000 7f0cadb3a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ca80b5bf0 0x7f0ca80adf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:45.671 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.661+0000 7f0cadb3a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ca8144390 0x7f0ca813c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:45.672 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.661+0000 7f0cadb3a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca80ae900 con 0x7f0ca8144390 2020-11-20T13:56:45.672 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.661+0000 7f0cadb3a700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca81523b0 con 0x7f0ca80b5bf0 2020-11-20T13:56:45.672 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.661+0000 7f0cadb3a700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca81520a0 con 0x7f0ca805fd70 2020-11-20T13:56:45.672 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.662+0000 7f0ca7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ca8144390 0x7f0ca813c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:45.672 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.662+0000 7f0ca7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ca8144390 0x7f0ca813c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55332/0 (socket says 172.21.15.40:55332) 2020-11-20T13:56:45.673 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.662+0000 7f0ca7fff700 1 -- 172.21.15.40:0/2444479358 learned_addr learned my addr 172.21.15.40:0/2444479358 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:45.673 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.662+0000 7f0ca77fe700 1 --2- 172.21.15.40:0/2444479358 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0ca805fd70 0x7f0ca813ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:45.673 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.662+0000 7f0ca6ffd700 1 --2- 172.21.15.40:0/2444479358 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ca80b5bf0 0x7f0ca80adf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:45.673 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.662+0000 7f0ca7fff700 1 -- 172.21.15.40:0/2444479358 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0ca805fd70 msgr2=0x7f0ca813ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:45.673 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.662+0000 7f0ca7fff700 1 --2- 172.21.15.40:0/2444479358 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0ca805fd70 0x7f0ca813ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:45.674 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.662+0000 7f0ca7fff700 1 -- 172.21.15.40:0/2444479358 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ca80b5bf0 msgr2=0x7f0ca80adf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:45.674 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.662+0000 7f0ca7fff700 1 --2- 172.21.15.40:0/2444479358 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ca80b5bf0 0x7f0ca80adf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:45.674 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.662+0000 7f0ca7fff700 1 -- 172.21.15.40:0/2444479358 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca80af560 con 0x7f0ca8144390 2020-11-20T13:56:45.674 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.662+0000 7f0ca7fff700 1 --2- 172.21.15.40:0/2444479358 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ca8144390 0x7f0ca813c0b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f0ca000a000 tx=0x7f0ca0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:45.674 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.662+0000 7f0ca4ff9700 1 -- 172.21.15.40:0/2444479358 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ca000b9e0 con 0x7f0ca8144390 2020-11-20T13:56:45.675 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.662+0000 7f0ca4ff9700 1 -- 172.21.15.40:0/2444479358 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ca000bb40 con 0x7f0ca8144390 2020-11-20T13:56:45.675 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.662+0000 7f0cadb3a700 1 -- 172.21.15.40:0/2444479358 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ca80ae470 con 0x7f0ca8144390 2020-11-20T13:56:45.675 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.662+0000 7f0ca4ff9700 1 -- 172.21.15.40:0/2444479358 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ca000bea0 con 0x7f0ca8144390 2020-11-20T13:56:45.675 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.662+0000 7f0cadb3a700 1 -- 172.21.15.40:0/2444479358 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ca813c7e0 con 0x7f0ca8144390 2020-11-20T13:56:45.675 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.663+0000 7f0cadb3a700 1 -- 172.21.15.40:0/2444479358 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ca8152210 con 0x7f0ca8144390 2020-11-20T13:56:45.676 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.663+0000 7f0ca4ff9700 1 -- 172.21.15.40:0/2444479358 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f0ca001c460 con 0x7f0ca8144390 2020-11-20T13:56:45.676 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.664+0000 7f0ca4ff9700 1 --2- 172.21.15.40:0/2444479358 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f0c880513c0 0x7f0c88053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:45.676 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.664+0000 7f0ca4ff9700 1 -- 172.21.15.40:0/2444479358 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f0ca0069760 con 0x7f0ca8144390 2020-11-20T13:56:45.676 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.664+0000 7f0ca77fe700 1 --2- 172.21.15.40:0/2444479358 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f0c880513c0 0x7f0c88053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:45.677 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.664+0000 7f0ca77fe700 1 --2- 172.21.15.40:0/2444479358 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f0c880513c0 0x7f0c88053880 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f0c9801c000 tx=0x7f0c98016000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:45.677 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.667+0000 7f0ca4ff9700 1 -- 172.21.15.40:0/2444479358 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f0ca8152210 con 0x7f0ca8144390 2020-11-20T13:56:45.829 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.828+0000 7f0cadb3a700 1 -- 172.21.15.40:0/2444479358 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0ca8152210 con 0x7f0ca8144390 2020-11-20T13:56:45.830 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.828+0000 7f0ca4ff9700 1 -- 172.21.15.40:0/2444479358 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+10 (secure 0 0 0) 0x7f0ca8152210 con 0x7f0ca8144390 2020-11-20T13:56:45.830 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2020-11-20T13:56:45.831 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.830+0000 7f0cadb3a700 1 -- 172.21.15.40:0/2444479358 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f0c880513c0 msgr2=0x7f0c88053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:45.831 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.830+0000 7f0cadb3a700 1 --2- 172.21.15.40:0/2444479358 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f0c880513c0 0x7f0c88053880 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f0c9801c000 tx=0x7f0c98016000).stop 2020-11-20T13:56:45.831 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.830+0000 7f0cadb3a700 1 -- 172.21.15.40:0/2444479358 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ca8144390 msgr2=0x7f0ca813c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:45.832 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.830+0000 7f0cadb3a700 1 --2- 172.21.15.40:0/2444479358 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ca8144390 0x7f0ca813c0b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f0ca000a000 tx=0x7f0ca0008040).stop 2020-11-20T13:56:45.832 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.830+0000 7f0cadb3a700 1 -- 172.21.15.40:0/2444479358 shutdown_connections 2020-11-20T13:56:45.832 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.830+0000 7f0cadb3a700 1 --2- 172.21.15.40:0/2444479358 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f0c880513c0 0x7f0c88053880 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:45.832 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.830+0000 7f0cadb3a700 1 --2- 172.21.15.40:0/2444479358 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0ca805fd70 0x7f0ca813ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:45.832 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.830+0000 7f0cadb3a700 1 --2- 172.21.15.40:0/2444479358 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ca8144390 0x7f0ca813c0b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:45.833 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.830+0000 7f0cadb3a700 1 --2- 172.21.15.40:0/2444479358 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ca80b5bf0 0x7f0ca80adf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:45.833 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.830+0000 7f0cadb3a700 1 -- 172.21.15.40:0/2444479358 >> 172.21.15.40:0/2444479358 conn(0x7f0ca80ba1d0 msgr2=0x7f0ca80b8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:45.833 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.830+0000 7f0cadb3a700 1 -- 172.21.15.40:0/2444479358 shutdown_connections 2020-11-20T13:56:45.833 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.830+0000 7f0cadb3a700 1 -- 172.21.15.40:0/2444479358 wait complete. 2020-11-20T13:56:45.841 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T13:56:45.841 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-20T13:56:45.955 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.954+0000 7fc6f1765700 1 Processor -- start 2020-11-20T13:56:45.955 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.954+0000 7fc6f1765700 1 -- start start 2020-11-20T13:56:45.957 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.955+0000 7fc6f1765700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc6ec0b88c0 0x7fc6ec0b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:45.958 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.955+0000 7fc6f1765700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc6ec09b530 0x7fc6ec09b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:45.958 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.955+0000 7fc6f1765700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6ec0b6be0 con 0x7fc6ec09be80 2020-11-20T13:56:45.958 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.955+0000 7fc6f1765700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6ec099000 con 0x7fc6ec09b530 2020-11-20T13:56:45.958 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.955+0000 7fc6f1765700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ec0b4250 con 0x7fc6ec0b88c0 2020-11-20T13:56:45.959 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.955+0000 7fc6ea7fc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc6ec09b530 0x7fc6ec09b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:45.959 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.955+0000 7fc6ea7fc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc6ec09b530 0x7fc6ec09b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55338/0 (socket says 172.21.15.40:55338) 2020-11-20T13:56:45.959 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.955+0000 7fc6ea7fc700 1 -- 172.21.15.40:0/1716801952 learned_addr learned my addr 172.21.15.40:0/1716801952 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:45.959 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.955+0000 7fc6eaffd700 1 --2- 172.21.15.40:0/1716801952 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc6ec0b88c0 0x7fc6ec0b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:45.960 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.955+0000 7fc6ea7fc700 1 -- 172.21.15.40:0/1716801952 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc6ec0b88c0 msgr2=0x7fc6ec0b8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:45.960 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.955+0000 7fc6ea7fc700 1 --2- 172.21.15.40:0/1716801952 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc6ec0b88c0 0x7fc6ec0b8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:45.960 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.955+0000 7fc6ea7fc700 1 -- 172.21.15.40:0/1716801952 >> v1:172.21.15.116:6789/0 conn(0x7fc6ec09be80 legacy=0x7fc6ec099270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:45.961 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.955+0000 7fc6ea7fc700 1 -- 172.21.15.40:0/1716801952 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ec0b4390 con 0x7fc6ec09b530 2020-11-20T13:56:45.961 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.955+0000 7fc6ea7fc700 1 --2- 172.21.15.40:0/1716801952 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6ec09b530 0x7fc6ec09b990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7fc6e0002f30 tx=0x7fc6e0006e80).ready entity=mon.0 client_cookie=93859932820582f6 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:45.961 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.955+0000 7fc6e9ffb700 1 -- 172.21.15.40:0/1716801952 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6e00097f0 con 0x7fc6ec09b530 2020-11-20T13:56:45.962 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.955+0000 7fc6e9ffb700 1 -- 172.21.15.40:0/1716801952 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e0009950 con 0x7fc6ec09b530 2020-11-20T13:56:45.962 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.955+0000 7fc6e9ffb700 1 -- 172.21.15.40:0/1716801952 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6e0009cb0 con 0x7fc6ec09b530 2020-11-20T13:56:45.962 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.955+0000 7fc6f1765700 1 -- 172.21.15.40:0/1716801952 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6ec09b530 msgr2=0x7fc6ec09b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:45.962 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.955+0000 7fc6f1765700 1 --2- 172.21.15.40:0/1716801952 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6ec09b530 0x7fc6ec09b990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7fc6e0002f30 tx=0x7fc6e0006e80).stop 2020-11-20T13:56:45.962 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.956+0000 7fc6f1765700 1 -- 172.21.15.40:0/1716801952 shutdown_connections 2020-11-20T13:56:45.963 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.956+0000 7fc6f1765700 1 --2- 172.21.15.40:0/1716801952 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc6ec0b88c0 0x7fc6ec0b8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:45.963 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.956+0000 7fc6f1765700 1 --2- 172.21.15.40:0/1716801952 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6ec09b530 0x7fc6ec09b990 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:45.963 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.956+0000 7fc6f1765700 1 -- 172.21.15.40:0/1716801952 >> 172.21.15.40:0/1716801952 conn(0x7fc6ec0ba380 msgr2=0x7fc6ec0b6350 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:45.963 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.956+0000 7fc6f1765700 1 -- 172.21.15.40:0/1716801952 shutdown_connections 2020-11-20T13:56:45.964 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.956+0000 7fc6f1765700 1 -- 172.21.15.40:0/1716801952 wait complete. 2020-11-20T13:56:45.966 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.957+0000 7fc6f1765700 1 Processor -- start 2020-11-20T13:56:45.966 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.957+0000 7fc6f1765700 1 -- start start 2020-11-20T13:56:45.967 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.958+0000 7fc6f1765700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc6ec09b530 0x7fc6ec0b3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:45.967 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.958+0000 7fc6f1765700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6ec09be80 0x7fc6ec140de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:45.967 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.958+0000 7fc6f1765700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6ec0b88c0 0x7fc6ec1412d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:45.967 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.958+0000 7fc6f1765700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ec09d8f0 con 0x7fc6ec09be80 2020-11-20T13:56:45.969 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.958+0000 7fc6f1765700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ec09db20 con 0x7fc6ec0b88c0 2020-11-20T13:56:45.969 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.958+0000 7fc6f1765700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ec147ec0 con 0x7fc6ec09b530 2020-11-20T13:56:45.969 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.958+0000 7fc6ea7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6ec09be80 0x7fc6ec140de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:45.969 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.958+0000 7fc6ea7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6ec09be80 0x7fc6ec140de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55344/0 (socket says 172.21.15.40:55344) 2020-11-20T13:56:45.970 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.958+0000 7fc6ea7fc700 1 -- 172.21.15.40:0/1138099220 learned_addr learned my addr 172.21.15.40:0/1138099220 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:45.970 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.958+0000 7fc6eb7fe700 1 --2- 172.21.15.40:0/1138099220 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6ec0b88c0 0x7fc6ec1412d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:45.970 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.958+0000 7fc6eaffd700 1 --2- 172.21.15.40:0/1138099220 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc6ec09b530 0x7fc6ec0b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:45.970 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.958+0000 7fc6ea7fc700 1 -- 172.21.15.40:0/1138099220 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc6ec09b530 msgr2=0x7fc6ec0b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:45.971 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.958+0000 7fc6ea7fc700 1 --2- 172.21.15.40:0/1138099220 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc6ec09b530 0x7fc6ec0b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:45.971 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.958+0000 7fc6ea7fc700 1 -- 172.21.15.40:0/1138099220 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6ec0b88c0 msgr2=0x7fc6ec1412d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:45.971 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.958+0000 7fc6ea7fc700 1 --2- 172.21.15.40:0/1138099220 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6ec0b88c0 0x7fc6ec1412d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:45.972 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.958+0000 7fc6ea7fc700 1 -- 172.21.15.40:0/1138099220 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ec1481d0 con 0x7fc6ec09be80 2020-11-20T13:56:45.972 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.958+0000 7fc6ea7fc700 1 --2- 172.21.15.40:0/1138099220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6ec09be80 0x7fc6ec140de0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7fc6e0002f30 tx=0x7fc6e0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:45.972 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.958+0000 7fc6dbfff700 1 -- 172.21.15.40:0/1138099220 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6e0011e70 con 0x7fc6ec09be80 2020-11-20T13:56:45.972 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.958+0000 7fc6dbfff700 1 -- 172.21.15.40:0/1138099220 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e0009c50 con 0x7fc6ec09be80 2020-11-20T13:56:45.973 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.958+0000 7fc6f1765700 1 -- 172.21.15.40:0/1138099220 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6ec1546b0 con 0x7fc6ec09be80 2020-11-20T13:56:45.973 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.958+0000 7fc6dbfff700 1 -- 172.21.15.40:0/1138099220 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6e001e650 con 0x7fc6ec09be80 2020-11-20T13:56:45.973 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.958+0000 7fc6f1765700 1 -- 172.21.15.40:0/1138099220 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6ec148fd0 con 0x7fc6ec09be80 2020-11-20T13:56:45.973 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.959+0000 7fc6f1765700 1 -- 172.21.15.40:0/1138099220 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6ec057050 con 0x7fc6ec09be80 2020-11-20T13:56:45.973 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.959+0000 7fc6dbfff700 1 -- 172.21.15.40:0/1138099220 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fc6e0025030 con 0x7fc6ec09be80 2020-11-20T13:56:45.974 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.960+0000 7fc6dbfff700 1 --2- 172.21.15.40:0/1138099220 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fc6c8051350 0x7fc6c8053810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:45.974 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.960+0000 7fc6dbfff700 1 -- 172.21.15.40:0/1138099220 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fc6e006a390 con 0x7fc6ec09be80 2020-11-20T13:56:45.974 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.960+0000 7fc6eaffd700 1 --2- 172.21.15.40:0/1138099220 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fc6c8051350 0x7fc6c8053810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:45.974 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.960+0000 7fc6eaffd700 1 --2- 172.21.15.40:0/1138099220 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fc6c8051350 0x7fc6c8053810 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fc6dc00c000 tx=0x7fc6dc008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:45.974 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:45.963+0000 7fc6dbfff700 1 -- 172.21.15.40:0/1138099220 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fc6e00425b0 con 0x7fc6ec09be80 2020-11-20T13:56:46.183 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:46.178+0000 7fc6f1765700 1 -- 172.21.15.40:0/1138099220 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6ec0b9690 con 0x7fc6ec09be80 2020-11-20T13:56:46.183 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:46.178+0000 7fc6dbfff700 1 -- 172.21.15.40:0/1138099220 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+2134 (secure 0 0 0) 0x7fc6e0023090 con 0x7fc6ec09be80 2020-11-20T13:56:46.183 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:46.180+0000 7fc6f1765700 1 -- 172.21.15.40:0/1138099220 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fc6c8051350 msgr2=0x7fc6c8053810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:46.184 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:46.180+0000 7fc6f1765700 1 --2- 172.21.15.40:0/1138099220 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fc6c8051350 0x7fc6c8053810 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fc6dc00c000 tx=0x7fc6dc008040).stop 2020-11-20T13:56:46.184 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:46.180+0000 7fc6f1765700 1 -- 172.21.15.40:0/1138099220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6ec09be80 msgr2=0x7fc6ec140de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:46.184 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:46.180+0000 7fc6f1765700 1 --2- 172.21.15.40:0/1138099220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6ec09be80 0x7fc6ec140de0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7fc6e0002f30 tx=0x7fc6e0015000).stop 2020-11-20T13:56:46.184 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:46.180+0000 7fc6f1765700 1 -- 172.21.15.40:0/1138099220 shutdown_connections 2020-11-20T13:56:46.184 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:46.180+0000 7fc6f1765700 1 --2- 172.21.15.40:0/1138099220 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fc6c8051350 0x7fc6c8053810 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:46.185 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:46.180+0000 7fc6f1765700 1 --2- 172.21.15.40:0/1138099220 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc6ec09b530 0x7fc6ec0b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:46.185 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:46.180+0000 7fc6f1765700 1 --2- 172.21.15.40:0/1138099220 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6ec0b88c0 0x7fc6ec1412d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:46.185 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:46.180+0000 7fc6f1765700 1 --2- 172.21.15.40:0/1138099220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6ec09be80 0x7fc6ec140de0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:46.185 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:46.180+0000 7fc6f1765700 1 -- 172.21.15.40:0/1138099220 >> 172.21.15.40:0/1138099220 conn(0x7fc6ec0ba380 msgr2=0x7fc6ec142ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:46.185 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:46.180+0000 7fc6f1765700 1 -- 172.21.15.40:0/1138099220 shutdown_connections 2020-11-20T13:56:46.186 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:46.180+0000 7fc6f1765700 1 -- 172.21.15.40:0/1138099220 wait complete. 2020-11-20T13:56:46.186 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 7 2020-11-20T13:56:46.186 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:56:46.186 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/2019740031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2019740031},{"type":"v1","addr":"172.21.15.116:6835","nonce":2019740031}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T13:56:25.428178+0000","modified":"2020-11-20T13:56:27.432735+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6833/584131564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":584131564},{"type":"v1","addr":"172.21.15.40:6833","nonce":584131564}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-20T13:56:46.395 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T13:56:25.428178+0000', 'modified': '2020-11-20T13:56:27.432735+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4263}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4263': {'gid': 4263, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.40:6833/584131564', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6832', 'nonce': 584131564}, {'type': 'v1', 'addr': '172.21.15.40:6833', 'nonce': 584131564}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T13:56:46.396 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T13:56:46.396 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T13:56:46.534 INFO:teuthology.orchestra.run.smithi040.stdout:{"cluster_fsid":"da1ed737-5174-430f-a3c9-56f6bbfad13c","whoami":0,"id":4263,"want_state":"up:active","state":"up:active","rank_uptime":21.095480930000001,"mdsmap_epoch":7,"osdmap_epoch":24,"osdmap_epoch_barrier":24,"uptime":27.410024502999999} 2020-11-20T13:56:46.535 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "da1ed737-5174-430f-a3c9-56f6bbfad13c", "id": 4263, "mdsmap_epoch": 7, "osdmap_epoch": 24, "osdmap_epoch_barrier": 24, "rank_uptime": 21.09548093, "state": "up:active", "uptime": 27.410024503, "want_state": "up:active", "whoami": 0 } 2020-11-20T13:56:46.536 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-11-20T13:56:46.536 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2020-11-20T13:56:46.536 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.2 at ubuntu@smithi116.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.2... 2020-11-20T13:56:46.536 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:46.537 INFO:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.2) 2020-11-20T13:56:46.555 INFO:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2020-11-20T13:56:46.714 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-20T13:56:46.715 INFO:teuthology.orchestra.run.smithi116:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-20T13:56:46.747 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:46.748 INFO:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2020-11-20T13:56:46.803 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-11-20T13:56:46.804 INFO:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.2 /home/ubuntu/cephtest/mnt.2) 2020-11-20T13:56:46.846 INFO:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2020-11-20T13:56:46.875 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-20T13:56:46.876 INFO:teuthology.orchestra.run.smithi116:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-20T13:56:46.923 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi116.stderr:2020-11-20T13:56:46.923+0000 7f8ad8c111c0 -1 init, newargv = 0x55f97310d280 newargc=9 2020-11-20T13:56:46.924 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi116.stderr:ceph-fuse[23880]: starting ceph client 2020-11-20T13:56:46.928 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi116.stderr:ceph-fuse[23880]: starting fuse 2020-11-20T13:56:46.945 INFO:teuthology.orchestra.run.smithi116.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-11-20T13:56:46.946 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-20T13:56:46.946 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:46.947 INFO:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2020-11-20T13:56:47.000 INFO:teuthology.orchestra.run.smithi116.stdout:47 2020-11-20T13:56:47.001 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2020-11-20T13:56:47.001 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:47.001 INFO:teuthology.orchestra.run.smithi116:> sudo python3 -c ' 2020-11-20T13:56:47.002 INFO:teuthology.orchestra.run.smithi116:> import glob 2020-11-20T13:56:47.002 INFO:teuthology.orchestra.run.smithi116:> import re 2020-11-20T13:56:47.002 INFO:teuthology.orchestra.run.smithi116:> import os 2020-11-20T13:56:47.003 INFO:teuthology.orchestra.run.smithi116:> import subprocess 2020-11-20T13:56:47.003 INFO:teuthology.orchestra.run.smithi116:> 2020-11-20T13:56:47.003 INFO:teuthology.orchestra.run.smithi116:> def find_socket(client_name): 2020-11-20T13:56:47.004 INFO:teuthology.orchestra.run.smithi116:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2020-11-20T13:56:47.004 INFO:teuthology.orchestra.run.smithi116:> files = glob.glob(asok_path) 2020-11-20T13:56:47.004 INFO:teuthology.orchestra.run.smithi116:> 2020-11-20T13:56:47.005 INFO:teuthology.orchestra.run.smithi116:> # Given a non-glob path, it better be there 2020-11-20T13:56:47.005 INFO:teuthology.orchestra.run.smithi116:> if "*" not in asok_path: 2020-11-20T13:56:47.006 INFO:teuthology.orchestra.run.smithi116:> assert(len(files) == 1) 2020-11-20T13:56:47.006 INFO:teuthology.orchestra.run.smithi116:> return files[0] 2020-11-20T13:56:47.006 INFO:teuthology.orchestra.run.smithi116:> 2020-11-20T13:56:47.007 INFO:teuthology.orchestra.run.smithi116:> for f in files: 2020-11-20T13:56:47.007 INFO:teuthology.orchestra.run.smithi116:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-11-20T13:56:47.007 INFO:teuthology.orchestra.run.smithi116:> if os.path.exists("/proc/{0}".format(pid)): 2020-11-20T13:56:47.008 INFO:teuthology.orchestra.run.smithi116:> return f 2020-11-20T13:56:47.008 INFO:teuthology.orchestra.run.smithi116:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-11-20T13:56:47.008 INFO:teuthology.orchestra.run.smithi116:> 2020-11-20T13:56:47.009 INFO:teuthology.orchestra.run.smithi116:> print(find_socket("client.2")) 2020-11-20T13:56:47.009 INFO:teuthology.orchestra.run.smithi116:> ' 2020-11-20T13:56:47.092 INFO:teuthology.orchestra.run.smithi116.stdout:/var/run/ceph/ceph-client.2.23880.asok 2020-11-20T13:56:47.094 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.23880.asok 2020-11-20T13:56:47.095 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:47.095 INFO:teuthology.orchestra.run.smithi116:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.23880.asok status 2020-11-20T13:56:47.198 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2020-11-20T13:56:47.199 INFO:teuthology.orchestra.run.smithi116.stdout: "metadata": { 2020-11-20T13:56:47.199 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_sha1": "a7b62340ad27b26642bb54ec916a97c7e1f07a34", 2020-11-20T13:56:47.199 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_version": "ceph version 15.2.5-588-ga7b62340 (a7b62340ad27b26642bb54ec916a97c7e1f07a34) octopus (stable)", 2020-11-20T13:56:47.200 INFO:teuthology.orchestra.run.smithi116.stdout: "entity_id": "2", 2020-11-20T13:56:47.200 INFO:teuthology.orchestra.run.smithi116.stdout: "hostname": "smithi116", 2020-11-20T13:56:47.200 INFO:teuthology.orchestra.run.smithi116.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2020-11-20T13:56:47.200 INFO:teuthology.orchestra.run.smithi116.stdout: "pid": "23880", 2020-11-20T13:56:47.200 INFO:teuthology.orchestra.run.smithi116.stdout: "root": "/" 2020-11-20T13:56:47.201 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T13:56:47.201 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_count": 0, 2020-11-20T13:56:47.201 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_pinned_count": 0, 2020-11-20T13:56:47.201 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 4648, 2020-11-20T13:56:47.201 INFO:teuthology.orchestra.run.smithi116.stdout: "inst": { 2020-11-20T13:56:47.202 INFO:teuthology.orchestra.run.smithi116.stdout: "name": { 2020-11-20T13:56:47.202 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "client", 2020-11-20T13:56:47.202 INFO:teuthology.orchestra.run.smithi116.stdout: "num": 4648 2020-11-20T13:56:47.202 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T13:56:47.202 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2020-11-20T13:56:47.203 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T13:56:47.203 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:0", 2020-11-20T13:56:47.203 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 3153514328 2020-11-20T13:56:47.203 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T13:56:47.203 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T13:56:47.204 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2020-11-20T13:56:47.204 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T13:56:47.204 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:0", 2020-11-20T13:56:47.205 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 3153514328 2020-11-20T13:56:47.205 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T13:56:47.205 INFO:teuthology.orchestra.run.smithi116.stdout: "inst_str": "client.4648 172.21.15.116:0/3153514328", 2020-11-20T13:56:47.206 INFO:teuthology.orchestra.run.smithi116.stdout: "addr_str": "172.21.15.116:0/3153514328", 2020-11-20T13:56:47.206 INFO:teuthology.orchestra.run.smithi116.stdout: "inode_count": 1, 2020-11-20T13:56:47.206 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_epoch": 7, 2020-11-20T13:56:47.206 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch": 24, 2020-11-20T13:56:47.207 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch_barrier": 0, 2020-11-20T13:56:47.207 INFO:teuthology.orchestra.run.smithi116.stdout: "blacklisted": false 2020-11-20T13:56:47.207 INFO:teuthology.orchestra.run.smithi116.stdout:} 2020-11-20T13:56:47.208 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-20T13:56:47.335 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.334+0000 7fd22ba82700 1 Processor -- start 2020-11-20T13:56:47.335 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.334+0000 7fd22ba82700 1 -- start start 2020-11-20T13:56:47.339 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.335+0000 7fd22ba82700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd22409af20 0x7fd2240b4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:47.339 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.335+0000 7fd22ba82700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd2240b81c0 0x7fd2240b4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:47.339 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.335+0000 7fd22ba82700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd22409bff0 con 0x7fd22409bc10 2020-11-20T13:56:47.339 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.335+0000 7fd22ba82700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2240b8740 con 0x7fd2240b81c0 2020-11-20T13:56:47.339 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.335+0000 7fd22ba82700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2240b89e0 con 0x7fd22409af20 2020-11-20T13:56:47.340 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.335+0000 7fd22981e700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd22409bc10 0x7fd2240aff40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45946/0 (socket says 172.21.15.40:45946) 2020-11-20T13:56:47.340 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.335+0000 7fd22981e700 1 -- 172.21.15.40:0/1585288243 learned_addr learned my addr 172.21.15.40:0/1585288243 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:47.340 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.335+0000 7fd22a01f700 1 --2- 172.21.15.40:0/1585288243 >> v2:172.21.15.116:3300/0 conn(0x7fd2240b81c0 0x7fd2240b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:47.340 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.335+0000 7fd22881c700 1 -- 172.21.15.40:0/1585288243 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1271930873 0 0) 0x7fd22409bff0 con 0x7fd22409bc10 2020-11-20T13:56:47.340 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.335+0000 7fd22881c700 1 -- 172.21.15.40:0/1585288243 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd20c003230 con 0x7fd22409bc10 2020-11-20T13:56:47.341 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.335+0000 7fd22901d700 1 --2- 172.21.15.40:0/1585288243 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd22409af20 0x7fd2240b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:47.341 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.335+0000 7fd22881c700 1 -- 172.21.15.40:0/1585288243 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1497909866 0 0) 0x7fd20c003230 con 0x7fd22409bc10 2020-11-20T13:56:47.341 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.335+0000 7fd22881c700 1 -- 172.21.15.40:0/1585288243 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd22409af20 msgr2=0x7fd2240b4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:47.341 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.335+0000 7fd22881c700 1 --2- 172.21.15.40:0/1585288243 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd22409af20 0x7fd2240b4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:47.342 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.335+0000 7fd22881c700 1 -- 172.21.15.40:0/1585288243 >> v2:172.21.15.116:3300/0 conn(0x7fd2240b81c0 msgr2=0x7fd2240b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:47.342 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.335+0000 7fd22881c700 1 --2- 172.21.15.40:0/1585288243 >> v2:172.21.15.116:3300/0 conn(0x7fd2240b81c0 0x7fd2240b4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:47.342 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.335+0000 7fd22881c700 1 -- 172.21.15.40:0/1585288243 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2240b8c80 con 0x7fd22409bc10 2020-11-20T13:56:47.342 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.335+0000 7fd22881c700 1 -- 172.21.15.40:0/1585288243 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2366654341 0 0) 0x7fd2180016d0 con 0x7fd22409bc10 2020-11-20T13:56:47.342 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.335+0000 7fd22881c700 1 -- 172.21.15.40:0/1585288243 >> v1:172.21.15.40:6789/0 conn(0x7fd22409bc10 legacy=0x7fd2240aff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:47.343 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.335+0000 7fd22881c700 1 --2- 172.21.15.40:0/1585288243 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd20c003720 0x7fd20c003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:47.343 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.335+0000 7fd22881c700 1 --2- 172.21.15.40:0/1585288243 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd20c0044d0 0x7fd20c004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:47.343 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.335+0000 7fd22881c700 1 --2- 172.21.15.40:0/1585288243 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd22409af20 0x7fd20c004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:47.343 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.335+0000 7fd22881c700 1 -- 172.21.15.40:0/1585288243 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2240b8740 con 0x7fd20c0044d0 2020-11-20T13:56:47.343 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.335+0000 7fd22881c700 1 -- 172.21.15.40:0/1585288243 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2240b89e0 con 0x7fd20c003720 2020-11-20T13:56:47.344 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.335+0000 7fd22881c700 1 -- 172.21.15.40:0/1585288243 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd20c004260 con 0x7fd22409af20 2020-11-20T13:56:47.344 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.336+0000 7fd22901d700 1 --2- 172.21.15.40:0/1585288243 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd20c0044d0 0x7fd20c004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:47.344 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.336+0000 7fd22981e700 1 --2- 172.21.15.40:0/1585288243 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd20c003720 0x7fd20c003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:47.344 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.336+0000 7fd22a01f700 1 --2- 172.21.15.40:0/1585288243 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd22409af20 0x7fd20c004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:47.344 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.336+0000 7fd22901d700 1 -- 172.21.15.40:0/1585288243 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd22409af20 msgr2=0x7fd20c004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:47.345 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.336+0000 7fd22901d700 1 --2- 172.21.15.40:0/1585288243 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd22409af20 0x7fd20c004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:47.345 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.336+0000 7fd22901d700 1 -- 172.21.15.40:0/1585288243 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd20c003720 msgr2=0x7fd20c003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:47.345 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.336+0000 7fd22901d700 1 --2- 172.21.15.40:0/1585288243 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd20c003720 0x7fd20c003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:47.345 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.336+0000 7fd22901d700 1 -- 172.21.15.40:0/1585288243 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd20c003230 con 0x7fd20c0044d0 2020-11-20T13:56:47.345 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.336+0000 7fd22981e700 1 --2- 172.21.15.40:0/1585288243 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd20c003720 0x7fd20c003b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:56:47.346 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.336+0000 7fd22901d700 1 --2- 172.21.15.40:0/1585288243 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd20c0044d0 0x7fd20c004950 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7fd22000c000 tx=0x7fd220008040).ready entity=mon.0 client_cookie=f7082679940f1b26 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:47.346 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.336+0000 7fd22881c700 1 -- 172.21.15.40:0/1585288243 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd22000e970 con 0x7fd20c0044d0 2020-11-20T13:56:47.346 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.336+0000 7fd22881c700 1 -- 172.21.15.40:0/1585288243 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd22000ead0 con 0x7fd20c0044d0 2020-11-20T13:56:47.346 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.336+0000 7fd22881c700 1 -- 172.21.15.40:0/1585288243 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd22000ee30 con 0x7fd20c0044d0 2020-11-20T13:56:47.346 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.336+0000 7fd22ba82700 1 -- 172.21.15.40:0/1585288243 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd20c0044d0 msgr2=0x7fd20c004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:47.347 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.336+0000 7fd22ba82700 1 --2- 172.21.15.40:0/1585288243 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd20c0044d0 0x7fd20c004950 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7fd22000c000 tx=0x7fd220008040).stop 2020-11-20T13:56:47.347 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.336+0000 7fd22a01f700 1 -- 172.21.15.40:0/1585288243 reap_dead start 2020-11-20T13:56:47.347 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.337+0000 7fd22ba82700 1 -- 172.21.15.40:0/1585288243 shutdown_connections 2020-11-20T13:56:47.347 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.337+0000 7fd22ba82700 1 -- 172.21.15.40:0/1585288243 >> 172.21.15.40:0/1585288243 conn(0x7fd2240ba380 msgr2=0x7fd2240b6350 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:47.347 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.337+0000 7fd22ba82700 1 -- 172.21.15.40:0/1585288243 shutdown_connections 2020-11-20T13:56:47.348 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.337+0000 7fd22ba82700 1 -- 172.21.15.40:0/1585288243 wait complete. 2020-11-20T13:56:47.348 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.338+0000 7fd22ba82700 1 Processor -- start 2020-11-20T13:56:47.348 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.338+0000 7fd22ba82700 1 -- start start 2020-11-20T13:56:47.351 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.338+0000 7fd22ba82700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd22409ef20 0x7fd22414b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:47.351 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.338+0000 7fd22ba82700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd22414d500 0x7fd224139470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:47.351 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.338+0000 7fd22ba82700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd224139960 0x7fd22413b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:47.351 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.338+0000 7fd22ba82700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22413cf00 con 0x7fd22414d500 2020-11-20T13:56:47.351 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.338+0000 7fd22ba82700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22413d1a0 con 0x7fd22409ef20 2020-11-20T13:56:47.352 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.338+0000 7fd22ba82700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd22413aec0 con 0x7fd224139960 2020-11-20T13:56:47.352 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.338+0000 7fd22901d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd22414d500 0x7fd224139470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:47.352 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.339+0000 7fd22901d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd22414d500 0x7fd224139470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55362/0 (socket says 172.21.15.40:55362) 2020-11-20T13:56:47.352 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.339+0000 7fd22901d700 1 -- 172.21.15.40:0/622627011 learned_addr learned my addr 172.21.15.40:0/622627011 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:47.352 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.339+0000 7fd22981e700 1 --2- 172.21.15.40:0/622627011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd22409ef20 0x7fd22414b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:47.353 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.339+0000 7fd22a01f700 1 --2- 172.21.15.40:0/622627011 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd224139960 0x7fd22413b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:47.353 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.339+0000 7fd22901d700 1 -- 172.21.15.40:0/622627011 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd224139960 msgr2=0x7fd22413b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:47.353 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.339+0000 7fd22901d700 1 --2- 172.21.15.40:0/622627011 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd224139960 0x7fd22413b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:47.353 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.339+0000 7fd22901d700 1 -- 172.21.15.40:0/622627011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd22409ef20 msgr2=0x7fd22414b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:47.354 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.339+0000 7fd22901d700 1 --2- 172.21.15.40:0/622627011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd22409ef20 0x7fd22414b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:47.354 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.339+0000 7fd22901d700 1 -- 172.21.15.40:0/622627011 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd22409f300 con 0x7fd22414d500 2020-11-20T13:56:47.354 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.339+0000 7fd22981e700 1 --2- 172.21.15.40:0/622627011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd22409ef20 0x7fd22414b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:47.354 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.339+0000 7fd22a01f700 1 --2- 172.21.15.40:0/622627011 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd224139960 0x7fd22413b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:47.354 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.339+0000 7fd22901d700 1 --2- 172.21.15.40:0/622627011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd22414d500 0x7fd224139470 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7fd22000c000 tx=0x7fd22001a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:47.355 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.339+0000 7fd216ffd700 1 -- 172.21.15.40:0/622627011 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd22001c620 con 0x7fd22414d500 2020-11-20T13:56:47.355 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.339+0000 7fd216ffd700 1 -- 172.21.15.40:0/622627011 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd22001c780 con 0x7fd22414d500 2020-11-20T13:56:47.355 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.339+0000 7fd22ba82700 1 -- 172.21.15.40:0/622627011 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd22413cc10 con 0x7fd22414d500 2020-11-20T13:56:47.355 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.339+0000 7fd216ffd700 1 -- 172.21.15.40:0/622627011 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd22001cae0 con 0x7fd22414d500 2020-11-20T13:56:47.355 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.339+0000 7fd22ba82700 1 -- 172.21.15.40:0/622627011 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd22414d980 con 0x7fd22414d500 2020-11-20T13:56:47.356 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.340+0000 7fd216ffd700 1 -- 172.21.15.40:0/622627011 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fd22002c070 con 0x7fd22414d500 2020-11-20T13:56:47.356 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.340+0000 7fd22ba82700 1 -- 172.21.15.40:0/622627011 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd224057050 con 0x7fd22414d500 2020-11-20T13:56:47.356 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.340+0000 7fd216ffd700 1 --2- 172.21.15.40:0/622627011 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fd204051320 0x7fd2040537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:47.356 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.340+0000 7fd216ffd700 1 -- 172.21.15.40:0/622627011 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fd22006add0 con 0x7fd22414d500 2020-11-20T13:56:47.356 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.341+0000 7fd22981e700 1 --2- 172.21.15.40:0/622627011 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fd204051320 0x7fd2040537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:47.357 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.341+0000 7fd22981e700 1 --2- 172.21.15.40:0/622627011 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fd204051320 0x7fd2040537e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fd218005000 tx=0x7fd218009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:47.357 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.344+0000 7fd216ffd700 1 -- 172.21.15.40:0/622627011 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fd220042ff0 con 0x7fd22414d500 2020-11-20T13:56:47.505 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.504+0000 7fd22ba82700 1 -- 172.21.15.40:0/622627011 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2240699a0 con 0x7fd22414d500 2020-11-20T13:56:47.505 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.504+0000 7fd216ffd700 1 -- 172.21.15.40:0/622627011 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16347 (secure 0 0 0) 0x7fd220016030 con 0x7fd22414d500 2020-11-20T13:56:47.506 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:56:47.506 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":24,"fsid":"da1ed737-5174-430f-a3c9-56f6bbfad13c","created":"2020-11-20T13:56:04.990651+0000","modified":"2020-11-20T13:56:25.428032+0000","last_up_change":"2020-11-20T13:56:13.893026+0000","last_in_change":"2020-11-20T13:56:13.893026+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T13:56:08.330017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T13:56:17.939684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T13:56:20.563740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T13:56:23.062623+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9ea4c667-8938-46d6-a5ca-092d55c1f79e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":22825},{"type":"v1","addr":"172.21.15.40:6818","nonce":22825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":22825},{"type":"v1","addr":"172.21.15.40:6822","nonce":22825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":22825},{"type":"v1","addr":"172.21.15.40:6830","nonce":22825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":22825},{"type":"v1","addr":"172.21.15.40:6826","nonce":22825}]},"public_addr":"172.21.15.40:6818/22825","cluster_addr":"172.21.15.40:6822/22825","heartbeat_back_addr":"172.21.15.40:6830/22825","heartbeat_front_addr":"172.21.15.40:6826/22825","state":["exists","up"]},{"osd":1,"uuid":"515708d9-5087-4e55-8d14-cdea1c00eacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":22823},{"type":"v1","addr":"172.21.15.40:6801","nonce":22823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":22823},{"type":"v1","addr":"172.21.15.40:6803","nonce":22823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":22823},{"type":"v1","addr":"172.21.15.40:6807","nonce":22823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":22823},{"type":"v1","addr":"172.21.15.40:6805","nonce":22823}]},"public_addr":"172.21.15.40:6801/22823","cluster_addr":"172.21.15.40:6803/22823","heartbeat_back_addr":"172.21.15.40:6807/22823","heartbeat_front_addr":"172.21.15.40:6805/22823","state":["exists","up"]},{"osd":2,"uuid":"4d86bca7-743e-4983-b35f-36676c405b36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":22824},{"type":"v1","addr":"172.21.15.40:6809","nonce":22824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":22824},{"type":"v1","addr":"172.21.15.40:6811","nonce":22824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":22824},{"type":"v1","addr":"172.21.15.40:6815","nonce":22824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":22824},{"type":"v1","addr":"172.21.15.40:6813","nonce":22824}]},"public_addr":"172.21.15.40:6809/22824","cluster_addr":"172.21.15.40:6811/22824","heartbeat_back_addr":"172.21.15.40:6815/22824","heartbeat_front_addr":"172.21.15.40:6813/22824","state":["exists","up"]},{"osd":3,"uuid":"f4dc726b-6b34-4e8b-8eeb-87060b58002d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":22826},{"type":"v1","addr":"172.21.15.40:6819","nonce":22826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":22826},{"type":"v1","addr":"172.21.15.40:6823","nonce":22826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":22826},{"type":"v1","addr":"172.21.15.40:6831","nonce":22826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":22826},{"type":"v1","addr":"172.21.15.40:6827","nonce":22826}]},"public_addr":"172.21.15.40:6819/22826","cluster_addr":"172.21.15.40:6823/22826","heartbeat_back_addr":"172.21.15.40:6831/22826","heartbeat_front_addr":"172.21.15.40:6827/22826","state":["exists","up"]},{"osd":4,"uuid":"25a6d53b-d1f0-4498-b7d5-ffa38661d403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":23181},{"type":"v1","addr":"172.21.15.116:6819","nonce":23181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":23181},{"type":"v1","addr":"172.21.15.116:6821","nonce":23181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":23181},{"type":"v1","addr":"172.21.15.116:6825","nonce":23181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":23181},{"type":"v1","addr":"172.21.15.116:6823","nonce":23181}]},"public_addr":"172.21.15.116:6819/23181","cluster_addr":"172.21.15.116:6821/23181","heartbeat_back_addr":"172.21.15.116:6825/23181","heartbeat_front_addr":"172.21.15.116:6823/23181","state":["exists","up"]},{"osd":5,"uuid":"7d7a5af8-229b-4351-9a21-a3d509cd4635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":23171},{"type":"v1","addr":"172.21.15.116:6803","nonce":23171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":23171},{"type":"v1","addr":"172.21.15.116:6805","nonce":23171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":23171},{"type":"v1","addr":"172.21.15.116:6809","nonce":23171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":23171},{"type":"v1","addr":"172.21.15.116:6807","nonce":23171}]},"public_addr":"172.21.15.116:6803/23171","cluster_addr":"172.21.15.116:6805/23171","heartbeat_back_addr":"172.21.15.116:6809/23171","heartbeat_front_addr":"172.21.15.116:6807/23171","state":["exists","up"]},{"osd":6,"uuid":"8247d248-a140-4e36-9796-9847a97df58d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":23172},{"type":"v1","addr":"172.21.15.116:6811","nonce":23172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":23172},{"type":"v1","addr":"172.21.15.116:6813","nonce":23172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":23172},{"type":"v1","addr":"172.21.15.116:6817","nonce":23172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":23172},{"type":"v1","addr":"172.21.15.116:6815","nonce":23172}]},"public_addr":"172.21.15.116:6811/23172","cluster_addr":"172.21.15.116:6813/23172","heartbeat_back_addr":"172.21.15.116:6817/23172","heartbeat_front_addr":"172.21.15.116:6815/23172","state":["exists","up"]},{"osd":7,"uuid":"472803c4-70b8-499d-b18a-71aaeec6aab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":23196},{"type":"v1","addr":"172.21.15.116:6827","nonce":23196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":23196},{"type":"v1","addr":"172.21.15.116:6829","nonce":23196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":23196},{"type":"v1","addr":"172.21.15.116:6833","nonce":23196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":23196},{"type":"v1","addr":"172.21.15.116:6831","nonce":23196}]},"public_addr":"172.21.15.116:6827/23196","cluster_addr":"172.21.15.116:6829/23196","heartbeat_back_addr":"172.21.15.116:6833/23196","heartbeat_front_addr":"172.21.15.116:6831/23196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.029357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:11.989797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.091097+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.014766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.070860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.071864+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.063400+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.109028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-20T13:56:47.507 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.506+0000 7fd22ba82700 1 -- 172.21.15.40:0/622627011 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fd204051320 msgr2=0x7fd2040537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:47.508 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.506+0000 7fd22ba82700 1 --2- 172.21.15.40:0/622627011 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fd204051320 0x7fd2040537e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fd218005000 tx=0x7fd218009040).stop 2020-11-20T13:56:47.508 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.506+0000 7fd22ba82700 1 -- 172.21.15.40:0/622627011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd22414d500 msgr2=0x7fd224139470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:47.509 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.506+0000 7fd22ba82700 1 --2- 172.21.15.40:0/622627011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd22414d500 0x7fd224139470 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7fd22000c000 tx=0x7fd22001a000).stop 2020-11-20T13:56:47.509 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.506+0000 7fd22ba82700 1 -- 172.21.15.40:0/622627011 shutdown_connections 2020-11-20T13:56:47.509 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.506+0000 7fd22ba82700 1 --2- 172.21.15.40:0/622627011 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fd204051320 0x7fd2040537e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:47.509 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.506+0000 7fd22ba82700 1 --2- 172.21.15.40:0/622627011 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd224139960 0x7fd22413b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:47.510 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.506+0000 7fd22ba82700 1 --2- 172.21.15.40:0/622627011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd22409ef20 0x7fd22414b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:47.510 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.506+0000 7fd22ba82700 1 --2- 172.21.15.40:0/622627011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd22414d500 0x7fd224139470 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:47.510 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.506+0000 7fd22ba82700 1 -- 172.21.15.40:0/622627011 >> 172.21.15.40:0/622627011 conn(0x7fd2240ba380 msgr2=0x7fd22413db10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:47.510 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.506+0000 7fd22ba82700 1 -- 172.21.15.40:0/622627011 shutdown_connections 2020-11-20T13:56:47.511 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.506+0000 7fd22ba82700 1 -- 172.21.15.40:0/622627011 wait complete. 2020-11-20T13:56:47.519 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T13:56:08.330017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T13:56:17.939684+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T13:56:20.563740+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T13:56:23.062623+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T13:56:47.520 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-20T13:56:47.638 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.636+0000 7fc92d438700 1 Processor -- start 2020-11-20T13:56:47.638 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.636+0000 7fc92d438700 1 -- start start 2020-11-20T13:56:47.640 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.637+0000 7fc92d438700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc928061f90 0x7fc92814b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:47.640 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.637+0000 7fc92d438700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc9280b6980 0x7fc9280b6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:47.640 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.637+0000 7fc92d438700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc928061310 con 0x7fc92805c510 2020-11-20T13:56:47.641 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.637+0000 7fc92d438700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9280615b0 con 0x7fc9280b6980 2020-11-20T13:56:47.641 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.637+0000 7fc92d438700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc928061850 con 0x7fc928061f90 2020-11-20T13:56:47.641 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.637+0000 7fc9277fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc9280b6980 0x7fc9280b6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:47.641 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.637+0000 7fc926ffd700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc92805c510 0x7fc92812d1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45964/0 (socket says 172.21.15.40:45964) 2020-11-20T13:56:47.642 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.638+0000 7fc926ffd700 1 -- 172.21.15.40:0/682697744 learned_addr learned my addr 172.21.15.40:0/682697744 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:47.642 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.638+0000 7fc9267fc700 1 --2- 172.21.15.40:0/682697744 >> v2:172.21.15.116:3300/0 conn(0x7fc928061f90 0x7fc92814b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:47.642 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.638+0000 7fc9277fe700 1 -- 172.21.15.40:0/682697744 >> v2:172.21.15.116:3300/0 conn(0x7fc928061f90 msgr2=0x7fc92814b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:47.642 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.638+0000 7fc9277fe700 1 --2- 172.21.15.40:0/682697744 >> v2:172.21.15.116:3300/0 conn(0x7fc928061f90 0x7fc92814b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:47.643 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.638+0000 7fc9277fe700 1 -- 172.21.15.40:0/682697744 >> v1:172.21.15.40:6789/0 conn(0x7fc92805c510 legacy=0x7fc92812d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:47.643 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.638+0000 7fc9277fe700 1 -- 172.21.15.40:0/682697744 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc928061af0 con 0x7fc9280b6980 2020-11-20T13:56:47.643 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.638+0000 7fc925ffb700 1 -- 172.21.15.40:0/682697744 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2463680268 0 0) 0x7fc928061310 con 0x7fc92805c510 2020-11-20T13:56:47.643 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.638+0000 7fc9277fe700 1 --2- 172.21.15.40:0/682697744 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9280b6980 0x7fc9280b6dd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7fc9200032b0 tx=0x7fc9200091a0).ready entity=mon.0 client_cookie=476b2c217c91360a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:47.643 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.638+0000 7fc925ffb700 1 -- 172.21.15.40:0/682697744 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc92000b9a0 con 0x7fc9280b6980 2020-11-20T13:56:47.644 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.638+0000 7fc925ffb700 1 -- 172.21.15.40:0/682697744 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc92000bb00 con 0x7fc9280b6980 2020-11-20T13:56:47.644 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.638+0000 7fc925ffb700 1 -- 172.21.15.40:0/682697744 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc92000bc70 con 0x7fc9280b6980 2020-11-20T13:56:47.644 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.638+0000 7fc92d438700 1 -- 172.21.15.40:0/682697744 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9280b6980 msgr2=0x7fc9280b6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:47.644 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.638+0000 7fc92d438700 1 --2- 172.21.15.40:0/682697744 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9280b6980 0x7fc9280b6dd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7fc9200032b0 tx=0x7fc9200091a0).stop 2020-11-20T13:56:47.645 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.639+0000 7fc92d438700 1 -- 172.21.15.40:0/682697744 shutdown_connections 2020-11-20T13:56:47.645 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.639+0000 7fc92d438700 1 --2- 172.21.15.40:0/682697744 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9280b6980 0x7fc9280b6dd0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:47.645 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.639+0000 7fc92d438700 1 --2- 172.21.15.40:0/682697744 >> v2:172.21.15.116:3300/0 conn(0x7fc928061f90 0x7fc92814b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:47.645 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.639+0000 7fc92d438700 1 -- 172.21.15.40:0/682697744 >> 172.21.15.40:0/682697744 conn(0x7fc9280bdb70 msgr2=0x7fc9280bdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:47.645 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.639+0000 7fc92d438700 1 -- 172.21.15.40:0/682697744 shutdown_connections 2020-11-20T13:56:47.646 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.639+0000 7fc92d438700 1 -- 172.21.15.40:0/682697744 wait complete. 2020-11-20T13:56:47.648 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.640+0000 7fc92d438700 1 Processor -- start 2020-11-20T13:56:47.648 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.640+0000 7fc92d438700 1 -- start start 2020-11-20T13:56:47.649 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.640+0000 7fc92d438700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc92805c510 0x7fc9280ad970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:47.649 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.640+0000 7fc92d438700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc928061f90 0x7fc9280b8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:47.649 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.640+0000 7fc92d438700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9280b6980 0x7fc928156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:47.649 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.640+0000 7fc92d438700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9280accf0 con 0x7fc9280b6980 2020-11-20T13:56:47.650 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.640+0000 7fc92d438700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9280acf90 con 0x7fc928061f90 2020-11-20T13:56:47.650 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.640+0000 7fc92d438700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9280ad230 con 0x7fc92805c510 2020-11-20T13:56:47.650 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.641+0000 7fc9277fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9280b6980 0x7fc928156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:47.650 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.641+0000 7fc9277fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9280b6980 0x7fc928156ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55376/0 (socket says 172.21.15.40:55376) 2020-11-20T13:56:47.650 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.641+0000 7fc9277fe700 1 -- 172.21.15.40:0/4282239431 learned_addr learned my addr 172.21.15.40:0/4282239431 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:47.651 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.641+0000 7fc9267fc700 1 --2- 172.21.15.40:0/4282239431 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc928061f90 0x7fc9280b8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:47.651 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.641+0000 7fc926ffd700 1 --2- 172.21.15.40:0/4282239431 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc92805c510 0x7fc9280ad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:47.651 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.641+0000 7fc9277fe700 1 -- 172.21.15.40:0/4282239431 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc92805c510 msgr2=0x7fc9280ad970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:47.651 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.641+0000 7fc9277fe700 1 --2- 172.21.15.40:0/4282239431 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc92805c510 0x7fc9280ad970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:47.651 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.641+0000 7fc9277fe700 1 -- 172.21.15.40:0/4282239431 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc928061f90 msgr2=0x7fc9280b8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:47.652 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.641+0000 7fc9277fe700 1 --2- 172.21.15.40:0/4282239431 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc928061f90 0x7fc9280b8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:47.652 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.641+0000 7fc9277fe700 1 -- 172.21.15.40:0/4282239431 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9280ad4d0 con 0x7fc9280b6980 2020-11-20T13:56:47.652 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.641+0000 7fc9267fc700 1 --2- 172.21.15.40:0/4282239431 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc928061f90 0x7fc9280b8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:47.652 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.641+0000 7fc926ffd700 1 --2- 172.21.15.40:0/4282239431 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc92805c510 0x7fc9280ad970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:47.652 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.641+0000 7fc9277fe700 1 --2- 172.21.15.40:0/4282239431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9280b6980 0x7fc928156ab0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7fc9200032b0 tx=0x7fc920016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:47.653 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.641+0000 7fc917fff700 1 -- 172.21.15.40:0/4282239431 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9200205f0 con 0x7fc9280b6980 2020-11-20T13:56:47.653 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.641+0000 7fc917fff700 1 -- 172.21.15.40:0/4282239431 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc920020750 con 0x7fc9280b6980 2020-11-20T13:56:47.653 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.641+0000 7fc92d438700 1 -- 172.21.15.40:0/4282239431 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9280b0a10 con 0x7fc9280b6980 2020-11-20T13:56:47.653 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.641+0000 7fc917fff700 1 -- 172.21.15.40:0/4282239431 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc920020ab0 con 0x7fc9280b6980 2020-11-20T13:56:47.653 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.641+0000 7fc92d438700 1 -- 172.21.15.40:0/4282239431 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9280b90e0 con 0x7fc9280b6980 2020-11-20T13:56:47.654 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.642+0000 7fc92d438700 1 -- 172.21.15.40:0/4282239431 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9280ad660 con 0x7fc9280b6980 2020-11-20T13:56:47.654 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.642+0000 7fc917fff700 1 -- 172.21.15.40:0/4282239431 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fc920027030 con 0x7fc9280b6980 2020-11-20T13:56:47.654 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.642+0000 7fc917fff700 1 --2- 172.21.15.40:0/4282239431 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fc9040512d0 0x7fc904053790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:47.654 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.643+0000 7fc917fff700 1 -- 172.21.15.40:0/4282239431 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fc92006a720 con 0x7fc9280b6980 2020-11-20T13:56:47.655 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.643+0000 7fc926ffd700 1 --2- 172.21.15.40:0/4282239431 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fc9040512d0 0x7fc904053790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:47.655 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.643+0000 7fc926ffd700 1 --2- 172.21.15.40:0/4282239431 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fc9040512d0 0x7fc904053790 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fc91c008000 tx=0x7fc91c009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:47.655 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.646+0000 7fc917fff700 1 -- 172.21.15.40:0/4282239431 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fc920042830 con 0x7fc9280b6980 2020-11-20T13:56:47.809 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.808+0000 7fc92d438700 1 -- 172.21.15.40:0/4282239431 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fc928057050 con 0x7fc9280b6980 2020-11-20T13:56:47.810 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.808+0000 7fc917fff700 1 -- 172.21.15.40:0/4282239431 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v24) v1 ==== 110+0+10 (secure 0 0 0) 0x7fc9200150b0 con 0x7fc9280b6980 2020-11-20T13:56:47.810 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2020-11-20T13:56:47.811 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.810+0000 7fc92d438700 1 -- 172.21.15.40:0/4282239431 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fc9040512d0 msgr2=0x7fc904053790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:47.811 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.810+0000 7fc92d438700 1 --2- 172.21.15.40:0/4282239431 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fc9040512d0 0x7fc904053790 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fc91c008000 tx=0x7fc91c009040).stop 2020-11-20T13:56:47.812 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.810+0000 7fc92d438700 1 -- 172.21.15.40:0/4282239431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9280b6980 msgr2=0x7fc928156ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:47.812 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.810+0000 7fc92d438700 1 --2- 172.21.15.40:0/4282239431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9280b6980 0x7fc928156ab0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7fc9200032b0 tx=0x7fc920016000).stop 2020-11-20T13:56:47.812 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.810+0000 7fc92d438700 1 -- 172.21.15.40:0/4282239431 shutdown_connections 2020-11-20T13:56:47.812 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.810+0000 7fc92d438700 1 --2- 172.21.15.40:0/4282239431 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fc9040512d0 0x7fc904053790 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:47.813 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.810+0000 7fc92d438700 1 --2- 172.21.15.40:0/4282239431 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc92805c510 0x7fc9280ad970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:47.813 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.810+0000 7fc92d438700 1 --2- 172.21.15.40:0/4282239431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9280b6980 0x7fc928156ab0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:47.813 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.810+0000 7fc92d438700 1 --2- 172.21.15.40:0/4282239431 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc928061f90 0x7fc9280b8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:47.813 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.810+0000 7fc92d438700 1 -- 172.21.15.40:0/4282239431 >> 172.21.15.40:0/4282239431 conn(0x7fc9280bdb70 msgr2=0x7fc92814a760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:47.813 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.810+0000 7fc92d438700 1 -- 172.21.15.40:0/4282239431 shutdown_connections 2020-11-20T13:56:47.814 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.810+0000 7fc92d438700 1 -- 172.21.15.40:0/4282239431 wait complete. 2020-11-20T13:56:47.821 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-20T13:56:47.938 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.937+0000 7f6046f95700 1 Processor -- start 2020-11-20T13:56:47.939 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.937+0000 7f6046f95700 1 -- start start 2020-11-20T13:56:47.942 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.938+0000 7f6046f95700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f60400b5bf0 0x7f604005f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:47.942 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.938+0000 7f6046f95700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6040144390 0x7f604005f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:47.942 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.938+0000 7f6046f95700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f604005fd70 0x7f60400b6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:47.943 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.938+0000 7f6046f95700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60400b5fd0 con 0x7f60400b5bf0 2020-11-20T13:56:47.943 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.938+0000 7f6046f95700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60400b05f0 con 0x7f6040144390 2020-11-20T13:56:47.943 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.938+0000 7f6046f95700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6040144870 con 0x7f604005fd70 2020-11-20T13:56:47.944 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.938+0000 7f6044d31700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f60400b5bf0 0x7f604005f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:47.944 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.938+0000 7f6044d31700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f60400b5bf0 0x7f604005f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55380/0 (socket says 172.21.15.40:55380) 2020-11-20T13:56:47.944 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.938+0000 7f6044d31700 1 -- 172.21.15.40:0/35800851 learned_addr learned my addr 172.21.15.40:0/35800851 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:47.944 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.938+0000 7f6045532700 1 --2- 172.21.15.40:0/35800851 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f604005fd70 0x7f60400b6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:47.945 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.938+0000 7f603ffff700 1 --2- 172.21.15.40:0/35800851 >> v2:172.21.15.116:3300/0 conn(0x7f6040144390 0x7f604005f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:47.945 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.938+0000 7f6044d31700 1 -- 172.21.15.40:0/35800851 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f604005fd70 msgr2=0x7f60400b6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:47.945 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.938+0000 7f6044d31700 1 --2- 172.21.15.40:0/35800851 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f604005fd70 0x7f60400b6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:47.946 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.938+0000 7f6044d31700 1 -- 172.21.15.40:0/35800851 >> v2:172.21.15.116:3300/0 conn(0x7f6040144390 msgr2=0x7f604005f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:47.946 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.938+0000 7f6044d31700 1 --2- 172.21.15.40:0/35800851 >> v2:172.21.15.116:3300/0 conn(0x7f6040144390 0x7f604005f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:47.946 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.938+0000 7f6044d31700 1 -- 172.21.15.40:0/35800851 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60400b0c30 con 0x7f60400b5bf0 2020-11-20T13:56:47.946 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.938+0000 7f6044d31700 1 --2- 172.21.15.40:0/35800851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60400b5bf0 0x7f604005f390 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f6030002f30 tx=0x7f6030006e20).ready entity=mon.0 client_cookie=98147878ef932584 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:47.946 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.939+0000 7f603f7fe700 1 -- 172.21.15.40:0/35800851 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f60300098c0 con 0x7f60400b5bf0 2020-11-20T13:56:47.947 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.939+0000 7f603f7fe700 1 -- 172.21.15.40:0/35800851 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6030009a20 con 0x7f60400b5bf0 2020-11-20T13:56:47.947 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.939+0000 7f603f7fe700 1 -- 172.21.15.40:0/35800851 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6030009d80 con 0x7f60400b5bf0 2020-11-20T13:56:47.947 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.939+0000 7f6046f95700 1 -- 172.21.15.40:0/35800851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60400b5bf0 msgr2=0x7f604005f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:47.947 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.939+0000 7f6046f95700 1 --2- 172.21.15.40:0/35800851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60400b5bf0 0x7f604005f390 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f6030002f30 tx=0x7f6030006e20).stop 2020-11-20T13:56:47.947 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.939+0000 7f6046f95700 1 -- 172.21.15.40:0/35800851 shutdown_connections 2020-11-20T13:56:47.948 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.939+0000 7f6046f95700 1 --2- 172.21.15.40:0/35800851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60400b5bf0 0x7f604005f390 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:47.948 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.939+0000 7f6046f95700 1 --2- 172.21.15.40:0/35800851 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f604005fd70 0x7f60400b6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:47.948 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.939+0000 7f6046f95700 1 --2- 172.21.15.40:0/35800851 >> v2:172.21.15.116:3300/0 conn(0x7f6040144390 0x7f604005f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:47.948 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.939+0000 7f6046f95700 1 -- 172.21.15.40:0/35800851 >> 172.21.15.40:0/35800851 conn(0x7f60400ba1d0 msgr2=0x7f60400b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:47.948 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.939+0000 7f6046f95700 1 -- 172.21.15.40:0/35800851 shutdown_connections 2020-11-20T13:56:47.949 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.939+0000 7f6046f95700 1 -- 172.21.15.40:0/35800851 wait complete. 2020-11-20T13:56:47.949 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.940+0000 7f6046f95700 1 Processor -- start 2020-11-20T13:56:47.949 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.940+0000 7f6046f95700 1 -- start start 2020-11-20T13:56:47.952 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.941+0000 7f6046f95700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f604005fd70 0x7f60400b2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:47.952 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.941+0000 7f6046f95700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60400b5bf0 0x7f604013ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:47.952 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.941+0000 7f6046f95700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6040144390 0x7f60400ad770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:47.952 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.941+0000 7f6046f95700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6040126910 con 0x7f6040144390 2020-11-20T13:56:47.952 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.941+0000 7f6046f95700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604013ef00 con 0x7f60400b5bf0 2020-11-20T13:56:47.953 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.941+0000 7f6046f95700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60400af1b0 con 0x7f604005fd70 2020-11-20T13:56:47.953 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.941+0000 7f6045532700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6040144390 0x7f60400ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:47.953 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.941+0000 7f6045532700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6040144390 0x7f60400ad770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55390/0 (socket says 172.21.15.40:55390) 2020-11-20T13:56:47.953 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.941+0000 7f6045532700 1 -- 172.21.15.40:0/1619768487 learned_addr learned my addr 172.21.15.40:0/1619768487 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:47.954 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.941+0000 7f6044d31700 1 --2- 172.21.15.40:0/1619768487 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f604005fd70 0x7f60400b2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:47.954 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.941+0000 7f603ffff700 1 --2- 172.21.15.40:0/1619768487 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60400b5bf0 0x7f604013ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:47.954 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.941+0000 7f6045532700 1 -- 172.21.15.40:0/1619768487 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f604005fd70 msgr2=0x7f60400b2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:47.954 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.941+0000 7f6045532700 1 --2- 172.21.15.40:0/1619768487 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f604005fd70 0x7f60400b2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:47.954 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.941+0000 7f6045532700 1 -- 172.21.15.40:0/1619768487 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60400b5bf0 msgr2=0x7f604013ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:47.955 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.941+0000 7f6045532700 1 --2- 172.21.15.40:0/1619768487 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60400b5bf0 0x7f604013ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:47.955 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.941+0000 7f6045532700 1 -- 172.21.15.40:0/1619768487 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60400af450 con 0x7f6040144390 2020-11-20T13:56:47.955 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.942+0000 7f6045532700 1 --2- 172.21.15.40:0/1619768487 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6040144390 0x7f60400ad770 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f603800a000 tx=0x7f6038008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:47.955 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.942+0000 7f603dffb700 1 -- 172.21.15.40:0/1619768487 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f603800b9e0 con 0x7f6040144390 2020-11-20T13:56:47.955 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.942+0000 7f603dffb700 1 -- 172.21.15.40:0/1619768487 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f603800bb40 con 0x7f6040144390 2020-11-20T13:56:47.956 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.942+0000 7f6046f95700 1 -- 172.21.15.40:0/1619768487 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60400af750 con 0x7f6040144390 2020-11-20T13:56:47.956 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.942+0000 7f603dffb700 1 -- 172.21.15.40:0/1619768487 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f603800bea0 con 0x7f6040144390 2020-11-20T13:56:47.956 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.942+0000 7f6046f95700 1 -- 172.21.15.40:0/1619768487 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60400adc60 con 0x7f6040144390 2020-11-20T13:56:47.956 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.943+0000 7f6046f95700 1 -- 172.21.15.40:0/1619768487 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60400af020 con 0x7f6040144390 2020-11-20T13:56:47.957 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.943+0000 7f603dffb700 1 -- 172.21.15.40:0/1619768487 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f603801c460 con 0x7f6040144390 2020-11-20T13:56:47.957 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.943+0000 7f603dffb700 1 --2- 172.21.15.40:0/1619768487 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f60200513c0 0x7f6020053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:47.958 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.943+0000 7f603dffb700 1 -- 172.21.15.40:0/1619768487 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f60380695d0 con 0x7f6040144390 2020-11-20T13:56:47.959 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.943+0000 7f6044d31700 1 --2- 172.21.15.40:0/1619768487 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f60200513c0 0x7f6020053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:47.959 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.943+0000 7f6044d31700 1 --2- 172.21.15.40:0/1619768487 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f60200513c0 0x7f6020053880 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f603001b000 tx=0x7f6030015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:47.959 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:47.947+0000 7f603dffb700 1 -- 172.21.15.40:0/1619768487 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f6038041790 con 0x7f6040144390 2020-11-20T13:56:48.108 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.106+0000 7f6046f95700 1 -- 172.21.15.40:0/1619768487 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6040057050 con 0x7f6040144390 2020-11-20T13:56:48.108 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.107+0000 7f603dffb700 1 -- 172.21.15.40:0/1619768487 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6038044db0 con 0x7f6040144390 2020-11-20T13:56:48.109 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2020-11-20T13:56:48.110 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.108+0000 7f6046f95700 1 -- 172.21.15.40:0/1619768487 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f60200513c0 msgr2=0x7f6020053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:48.110 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.108+0000 7f6046f95700 1 --2- 172.21.15.40:0/1619768487 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f60200513c0 0x7f6020053880 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f603001b000 tx=0x7f6030015000).stop 2020-11-20T13:56:48.110 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.108+0000 7f6046f95700 1 -- 172.21.15.40:0/1619768487 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6040144390 msgr2=0x7f60400ad770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:48.110 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.108+0000 7f6046f95700 1 --2- 172.21.15.40:0/1619768487 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6040144390 0x7f60400ad770 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f603800a000 tx=0x7f6038008040).stop 2020-11-20T13:56:48.111 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.108+0000 7f6046f95700 1 -- 172.21.15.40:0/1619768487 shutdown_connections 2020-11-20T13:56:48.111 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.108+0000 7f6046f95700 1 --2- 172.21.15.40:0/1619768487 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f60200513c0 0x7f6020053880 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.111 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.108+0000 7f6046f95700 1 --2- 172.21.15.40:0/1619768487 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f604005fd70 0x7f60400b2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.111 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.108+0000 7f6046f95700 1 --2- 172.21.15.40:0/1619768487 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6040144390 0x7f60400ad770 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.111 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.108+0000 7f6046f95700 1 --2- 172.21.15.40:0/1619768487 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60400b5bf0 0x7f604013ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.112 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.108+0000 7f6046f95700 1 -- 172.21.15.40:0/1619768487 >> 172.21.15.40:0/1619768487 conn(0x7f60400ba1d0 msgr2=0x7f60400b7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:48.112 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.108+0000 7f6046f95700 1 -- 172.21.15.40:0/1619768487 shutdown_connections 2020-11-20T13:56:48.112 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.109+0000 7f6046f95700 1 -- 172.21.15.40:0/1619768487 wait complete. 2020-11-20T13:56:48.120 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-20T13:56:48.236 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.235+0000 7fbb80370700 1 Processor -- start 2020-11-20T13:56:48.236 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.235+0000 7fbb80370700 1 -- start start 2020-11-20T13:56:48.238 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.236+0000 7fbb80370700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbb780b5bf0 0x7fbb7805f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:48.239 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.236+0000 7fbb80370700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fbb78144390 0x7fbb7805f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:48.239 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.236+0000 7fbb80370700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbb7805fd70 0x7fbb780b6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:48.239 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.236+0000 7fbb80370700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb780b5fd0 con 0x7fbb780b5bf0 2020-11-20T13:56:48.239 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.236+0000 7fbb80370700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb780b05f0 con 0x7fbb78144390 2020-11-20T13:56:48.239 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.236+0000 7fbb80370700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb78144870 con 0x7fbb7805fd70 2020-11-20T13:56:48.240 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.236+0000 7fbb7e10c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbb780b5bf0 0x7fbb7805f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:48.240 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.236+0000 7fbb7e10c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbb780b5bf0 0x7fbb7805f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55394/0 (socket says 172.21.15.40:55394) 2020-11-20T13:56:48.240 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.236+0000 7fbb7e10c700 1 -- 172.21.15.40:0/4092671203 learned_addr learned my addr 172.21.15.40:0/4092671203 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:48.240 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.236+0000 7fbb7e90d700 1 --2- 172.21.15.40:0/4092671203 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbb7805fd70 0x7fbb780b6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:48.240 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.236+0000 7fbb7d90b700 1 --2- 172.21.15.40:0/4092671203 >> v2:172.21.15.116:3300/0 conn(0x7fbb78144390 0x7fbb7805f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:48.241 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.236+0000 7fbb7e10c700 1 -- 172.21.15.40:0/4092671203 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbb7805fd70 msgr2=0x7fbb780b6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:48.241 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.236+0000 7fbb7e10c700 1 --2- 172.21.15.40:0/4092671203 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbb7805fd70 0x7fbb780b6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.241 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.236+0000 7fbb7e10c700 1 -- 172.21.15.40:0/4092671203 >> v2:172.21.15.116:3300/0 conn(0x7fbb78144390 msgr2=0x7fbb7805f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:48.241 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.236+0000 7fbb7e10c700 1 --2- 172.21.15.40:0/4092671203 >> v2:172.21.15.116:3300/0 conn(0x7fbb78144390 0x7fbb7805f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.241 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.236+0000 7fbb7e10c700 1 -- 172.21.15.40:0/4092671203 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb780b0c30 con 0x7fbb780b5bf0 2020-11-20T13:56:48.242 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.236+0000 7fbb7d90b700 1 --2- 172.21.15.40:0/4092671203 >> v2:172.21.15.116:3300/0 conn(0x7fbb78144390 0x7fbb7805f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:48.242 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.236+0000 7fbb7e90d700 1 --2- 172.21.15.40:0/4092671203 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbb7805fd70 0x7fbb780b6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:48.242 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.236+0000 7fbb7e10c700 1 --2- 172.21.15.40:0/4092671203 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb780b5bf0 0x7fbb7805f390 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7fbb780b2780 tx=0x7fbb74006e20).ready entity=mon.0 client_cookie=e0c8c6de37d0cee1 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:48.242 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.236+0000 7fbb7d10a700 1 -- 172.21.15.40:0/4092671203 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb74009880 con 0x7fbb780b5bf0 2020-11-20T13:56:48.242 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.236+0000 7fbb7d10a700 1 -- 172.21.15.40:0/4092671203 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb740099e0 con 0x7fbb780b5bf0 2020-11-20T13:56:48.243 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.236+0000 7fbb7d10a700 1 -- 172.21.15.40:0/4092671203 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb74009d40 con 0x7fbb780b5bf0 2020-11-20T13:56:48.243 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.236+0000 7fbb80370700 1 -- 172.21.15.40:0/4092671203 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb780b5bf0 msgr2=0x7fbb7805f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:48.243 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.236+0000 7fbb80370700 1 --2- 172.21.15.40:0/4092671203 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb780b5bf0 0x7fbb7805f390 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7fbb780b2780 tx=0x7fbb74006e20).stop 2020-11-20T13:56:48.243 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.237+0000 7fbb80370700 1 -- 172.21.15.40:0/4092671203 shutdown_connections 2020-11-20T13:56:48.244 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.237+0000 7fbb80370700 1 --2- 172.21.15.40:0/4092671203 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb780b5bf0 0x7fbb7805f390 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.244 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.237+0000 7fbb80370700 1 --2- 172.21.15.40:0/4092671203 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbb7805fd70 0x7fbb780b6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.244 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.237+0000 7fbb80370700 1 --2- 172.21.15.40:0/4092671203 >> v2:172.21.15.116:3300/0 conn(0x7fbb78144390 0x7fbb7805f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.244 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.237+0000 7fbb80370700 1 -- 172.21.15.40:0/4092671203 >> 172.21.15.40:0/4092671203 conn(0x7fbb780ba1d0 msgr2=0x7fbb780b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:48.245 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.237+0000 7fbb80370700 1 -- 172.21.15.40:0/4092671203 shutdown_connections 2020-11-20T13:56:48.245 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.237+0000 7fbb80370700 1 -- 172.21.15.40:0/4092671203 wait complete. 2020-11-20T13:56:48.248 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.238+0000 7fbb80370700 1 Processor -- start 2020-11-20T13:56:48.249 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.238+0000 7fbb80370700 1 -- start start 2020-11-20T13:56:48.249 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.239+0000 7fbb80370700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbb7805fd70 0x7fbb78062bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:48.249 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.239+0000 7fbb80370700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbb780b5bf0 0x7fbb7813a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:48.250 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.239+0000 7fbb80370700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb78144390 0x7fbb7813cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:48.250 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.239+0000 7fbb80370700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7813a070 con 0x7fbb78144390 2020-11-20T13:56:48.250 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.239+0000 7fbb80370700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7813a310 con 0x7fbb7805fd70 2020-11-20T13:56:48.250 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.239+0000 7fbb80370700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7813d420 con 0x7fbb780b5bf0 2020-11-20T13:56:48.251 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.239+0000 7fbb7e90d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb78144390 0x7fbb7813cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:48.251 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.239+0000 7fbb7e90d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb78144390 0x7fbb7813cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55404/0 (socket says 172.21.15.40:55404) 2020-11-20T13:56:48.251 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.239+0000 7fbb7e90d700 1 -- 172.21.15.40:0/4061888192 learned_addr learned my addr 172.21.15.40:0/4061888192 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:48.251 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.239+0000 7fbb7d90b700 1 --2- 172.21.15.40:0/4061888192 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbb780b5bf0 0x7fbb7813a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:48.251 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.239+0000 7fbb7e10c700 1 --2- 172.21.15.40:0/4061888192 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbb7805fd70 0x7fbb78062bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.239+0000 7fbb7e90d700 1 -- 172.21.15.40:0/4061888192 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbb780b5bf0 msgr2=0x7fbb7813a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.239+0000 7fbb7e90d700 1 --2- 172.21.15.40:0/4061888192 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbb780b5bf0 0x7fbb7813a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.239+0000 7fbb7e90d700 1 -- 172.21.15.40:0/4061888192 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbb7805fd70 msgr2=0x7fbb78062bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.239+0000 7fbb7e90d700 1 --2- 172.21.15.40:0/4061888192 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbb7805fd70 0x7fbb78062bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.239+0000 7fbb7e90d700 1 -- 172.21.15.40:0/4061888192 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb781386b0 con 0x7fbb78144390 2020-11-20T13:56:48.253 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.239+0000 7fbb7e90d700 1 --2- 172.21.15.40:0/4061888192 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb78144390 0x7fbb7813cf30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7fbb7000a000 tx=0x7fbb70008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:48.253 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.239+0000 7fbb677fe700 1 -- 172.21.15.40:0/4061888192 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb7000c7e0 con 0x7fbb78144390 2020-11-20T13:56:48.253 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.239+0000 7fbb677fe700 1 -- 172.21.15.40:0/4061888192 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb7000c940 con 0x7fbb78144390 2020-11-20T13:56:48.253 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.239+0000 7fbb80370700 1 -- 172.21.15.40:0/4061888192 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb78138960 con 0x7fbb78144390 2020-11-20T13:56:48.254 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.239+0000 7fbb677fe700 1 -- 172.21.15.40:0/4061888192 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb7000cca0 con 0x7fbb78144390 2020-11-20T13:56:48.254 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.239+0000 7fbb80370700 1 -- 172.21.15.40:0/4061888192 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb78134ef0 con 0x7fbb78144390 2020-11-20T13:56:48.254 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.241+0000 7fbb677fe700 1 -- 172.21.15.40:0/4061888192 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fbb7001d460 con 0x7fbb78144390 2020-11-20T13:56:48.254 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.241+0000 7fbb80370700 1 -- 172.21.15.40:0/4061888192 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb78057050 con 0x7fbb78144390 2020-11-20T13:56:48.254 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.241+0000 7fbb677fe700 1 --2- 172.21.15.40:0/4061888192 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fbb580513c0 0x7fbb58053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:48.255 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.241+0000 7fbb677fe700 1 -- 172.21.15.40:0/4061888192 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fbb700693a0 con 0x7fbb78144390 2020-11-20T13:56:48.255 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.241+0000 7fbb7e10c700 1 --2- 172.21.15.40:0/4061888192 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fbb580513c0 0x7fbb58053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:48.256 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.241+0000 7fbb7e10c700 1 --2- 172.21.15.40:0/4061888192 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fbb580513c0 0x7fbb58053880 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fbb7401b000 tx=0x7fbb74015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:48.256 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.244+0000 7fbb677fe700 1 -- 172.21.15.40:0/4061888192 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fbb70041560 con 0x7fbb78144390 2020-11-20T13:56:48.404 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.403+0000 7fbb80370700 1 -- 172.21.15.40:0/4061888192 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbb78138900 con 0x7fbb78144390 2020-11-20T13:56:48.404 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.403+0000 7fbb677fe700 1 -- 172.21.15.40:0/4061888192 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+10 (secure 0 0 0) 0x7fbb7000ce00 con 0x7fbb78144390 2020-11-20T13:56:48.405 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2020-11-20T13:56:48.406 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.404+0000 7fbb80370700 1 -- 172.21.15.40:0/4061888192 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fbb580513c0 msgr2=0x7fbb58053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:48.406 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.404+0000 7fbb80370700 1 --2- 172.21.15.40:0/4061888192 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fbb580513c0 0x7fbb58053880 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fbb7401b000 tx=0x7fbb74015000).stop 2020-11-20T13:56:48.406 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.404+0000 7fbb80370700 1 -- 172.21.15.40:0/4061888192 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb78144390 msgr2=0x7fbb7813cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:48.407 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.404+0000 7fbb80370700 1 --2- 172.21.15.40:0/4061888192 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb78144390 0x7fbb7813cf30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7fbb7000a000 tx=0x7fbb70008040).stop 2020-11-20T13:56:48.407 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.405+0000 7fbb80370700 1 -- 172.21.15.40:0/4061888192 shutdown_connections 2020-11-20T13:56:48.407 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.405+0000 7fbb80370700 1 --2- 172.21.15.40:0/4061888192 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fbb580513c0 0x7fbb58053880 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.407 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.405+0000 7fbb80370700 1 --2- 172.21.15.40:0/4061888192 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb78144390 0x7fbb7813cf30 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.407 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.405+0000 7fbb80370700 1 --2- 172.21.15.40:0/4061888192 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbb7805fd70 0x7fbb78062bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.408 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.405+0000 7fbb80370700 1 --2- 172.21.15.40:0/4061888192 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbb780b5bf0 0x7fbb7813a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.408 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.405+0000 7fbb80370700 1 -- 172.21.15.40:0/4061888192 >> 172.21.15.40:0/4061888192 conn(0x7fbb780ba1d0 msgr2=0x7fbb78146350 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:48.408 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.405+0000 7fbb80370700 1 -- 172.21.15.40:0/4061888192 shutdown_connections 2020-11-20T13:56:48.408 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.405+0000 7fbb80370700 1 -- 172.21.15.40:0/4061888192 wait complete. 2020-11-20T13:56:48.416 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-20T13:56:48.533 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.532+0000 7f88cf52b700 1 Processor -- start 2020-11-20T13:56:48.533 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.532+0000 7f88cf52b700 1 -- start start 2020-11-20T13:56:48.537 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.533+0000 7f88cf52b700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f88c80b31f0 0x7f88c8144bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:48.537 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.533+0000 7f88cf52b700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88c80b0d10 con 0x7f88c80b1450 2020-11-20T13:56:48.537 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.533+0000 7f88cf52b700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88c80b0fb0 con 0x7f88c80b6c80 2020-11-20T13:56:48.537 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.533+0000 7f88cf52b700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88c80b7060 con 0x7f88c80b31f0 2020-11-20T13:56:48.538 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.533+0000 7f88cdac8700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f88c80b1450 0x7f88c812ce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46006/0 (socket says 172.21.15.40:46006) 2020-11-20T13:56:48.538 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.533+0000 7f88cdac8700 1 -- 172.21.15.40:0/2918797807 learned_addr learned my addr 172.21.15.40:0/2918797807 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:48.538 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.533+0000 7f88c7fff700 1 -- 172.21.15.40:0/2918797807 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3090412471 0 0) 0x7f88c80b0d10 con 0x7f88c80b1450 2020-11-20T13:56:48.539 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.533+0000 7f88c7fff700 1 -- 172.21.15.40:0/2918797807 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88b4003230 con 0x7f88c80b1450 2020-11-20T13:56:48.539 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.533+0000 7f88ccac6700 1 --2- 172.21.15.40:0/2918797807 >> v2:172.21.15.116:3300/0 conn(0x7f88c80b31f0 0x7f88c8144bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:48.539 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.533+0000 7f88c7fff700 1 -- 172.21.15.40:0/2918797807 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3629229730 0 0) 0x7f88b4003230 con 0x7f88c80b1450 2020-11-20T13:56:48.539 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.533+0000 7f88c7fff700 1 -- 172.21.15.40:0/2918797807 >> v2:172.21.15.116:3300/0 conn(0x7f88c80b31f0 msgr2=0x7f88c8144bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:48.540 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.533+0000 7f88c7fff700 1 --2- 172.21.15.40:0/2918797807 >> v2:172.21.15.116:3300/0 conn(0x7f88c80b31f0 0x7f88c8144bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.540 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.533+0000 7f88c7fff700 1 -- 172.21.15.40:0/2918797807 >> v1:172.21.15.116:6789/0 conn(0x7f88c80b6c80 legacy=0x7f88c80b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:48.540 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.533+0000 7f88c7fff700 1 -- 172.21.15.40:0/2918797807 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c80b4800 con 0x7f88c80b1450 2020-11-20T13:56:48.540 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.533+0000 7f88c7fff700 1 -- 172.21.15.40:0/2918797807 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2366654341 0 0) 0x7f88b80016d0 con 0x7f88c80b1450 2020-11-20T13:56:48.540 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.533+0000 7f88c7fff700 1 -- 172.21.15.40:0/2918797807 >> v1:172.21.15.40:6789/0 conn(0x7f88c80b1450 legacy=0x7f88c812ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:48.541 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.533+0000 7f88c7fff700 1 --2- 172.21.15.40:0/2918797807 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88b4003720 0x7f88b4003b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:48.541 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.533+0000 7f88c7fff700 1 --2- 172.21.15.40:0/2918797807 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f88b40044d0 0x7f88b4004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:48.541 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.533+0000 7f88c7fff700 1 --2- 172.21.15.40:0/2918797807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88b4004e40 0x7f88b400b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:48.541 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.533+0000 7f88c7fff700 1 -- 172.21.15.40:0/2918797807 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c80b7060 con 0x7f88b4004e40 2020-11-20T13:56:48.541 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.533+0000 7f88c7fff700 1 -- 172.21.15.40:0/2918797807 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b400bc10 con 0x7f88b4003720 2020-11-20T13:56:48.542 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.533+0000 7f88c7fff700 1 -- 172.21.15.40:0/2918797807 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88b400bd50 con 0x7f88b40044d0 2020-11-20T13:56:48.542 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.533+0000 7f88cdac8700 1 --2- 172.21.15.40:0/2918797807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88b4004e40 0x7f88b400b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:48.542 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.534+0000 7f88cd2c7700 1 --2- 172.21.15.40:0/2918797807 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88b4003720 0x7f88b4003b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:48.542 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.534+0000 7f88ccac6700 1 --2- 172.21.15.40:0/2918797807 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f88b40044d0 0x7f88b4004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:48.542 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.534+0000 7f88cdac8700 1 -- 172.21.15.40:0/2918797807 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f88b40044d0 msgr2=0x7f88b4004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:48.543 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.534+0000 7f88cdac8700 1 --2- 172.21.15.40:0/2918797807 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f88b40044d0 0x7f88b4004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.543 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.534+0000 7f88cdac8700 1 -- 172.21.15.40:0/2918797807 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88b4003720 msgr2=0x7f88b4003b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:48.543 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.534+0000 7f88cdac8700 1 --2- 172.21.15.40:0/2918797807 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88b4003720 0x7f88b4003b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.543 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.534+0000 7f88cdac8700 1 -- 172.21.15.40:0/2918797807 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b4003230 con 0x7f88b4004e40 2020-11-20T13:56:48.543 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.534+0000 7f88cdac8700 1 -- 172.21.15.40:0/2918797807 reap_dead start 2020-11-20T13:56:48.544 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.534+0000 7f88cdac8700 1 --2- 172.21.15.40:0/2918797807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88b4004e40 0x7f88b400b720 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f88b80013e0 tx=0x7f88c80b1140).ready entity=mon.0 client_cookie=4e09be5d0b3bf487 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:48.544 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.534+0000 7f88c7fff700 1 -- 172.21.15.40:0/2918797807 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88b80070d0 con 0x7f88b4004e40 2020-11-20T13:56:48.544 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.534+0000 7f88c7fff700 1 -- 172.21.15.40:0/2918797807 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b8007230 con 0x7f88b4004e40 2020-11-20T13:56:48.544 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.534+0000 7f88c7fff700 1 -- 172.21.15.40:0/2918797807 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88b80055e0 con 0x7f88b4004e40 2020-11-20T13:56:48.544 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.534+0000 7f88cf52b700 1 -- 172.21.15.40:0/2918797807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88b4004e40 msgr2=0x7f88b400b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:48.545 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.534+0000 7f88cf52b700 1 --2- 172.21.15.40:0/2918797807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88b4004e40 0x7f88b400b720 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f88b80013e0 tx=0x7f88c80b1140).stop 2020-11-20T13:56:48.545 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.535+0000 7f88cf52b700 1 -- 172.21.15.40:0/2918797807 shutdown_connections 2020-11-20T13:56:48.545 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.535+0000 7f88cf52b700 1 --2- 172.21.15.40:0/2918797807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88b4004e40 0x7f88b400b720 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.547 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.535+0000 7f88cf52b700 1 -- 172.21.15.40:0/2918797807 >> 172.21.15.40:0/2918797807 conn(0x7f88c80ba1d0 msgr2=0x7f88c80b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:48.547 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.535+0000 7f88cf52b700 1 -- 172.21.15.40:0/2918797807 shutdown_connections 2020-11-20T13:56:48.548 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.535+0000 7f88cf52b700 1 -- 172.21.15.40:0/2918797807 wait complete. 2020-11-20T13:56:48.548 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.536+0000 7f88cf52b700 1 Processor -- start 2020-11-20T13:56:48.548 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.536+0000 7f88cf52b700 1 -- start start 2020-11-20T13:56:48.550 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.536+0000 7f88cf52b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88b4004e40 0x7f88c812b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:48.551 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.536+0000 7f88cf52b700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f88c80accf0 0x7f88c8143260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:48.551 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.536+0000 7f88cf52b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88c80ad170 0x7f88c8143750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:48.551 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.536+0000 7f88cf52b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c8143c40 con 0x7f88c80ad170 2020-11-20T13:56:48.551 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.536+0000 7f88cf52b700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c80ad600 con 0x7f88b4004e40 2020-11-20T13:56:48.551 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.536+0000 7f88cf52b700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88c80ad8a0 con 0x7f88c80accf0 2020-11-20T13:56:48.552 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.536+0000 7f88cdac8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88c80ad170 0x7f88c8143750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:48.552 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.536+0000 7f88cdac8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88c80ad170 0x7f88c8143750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55424/0 (socket says 172.21.15.40:55424) 2020-11-20T13:56:48.552 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.536+0000 7f88cdac8700 1 -- 172.21.15.40:0/2295086367 learned_addr learned my addr 172.21.15.40:0/2295086367 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:48.552 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.537+0000 7f88ccac6700 1 --2- 172.21.15.40:0/2295086367 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f88c80accf0 0x7f88c8143260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:48.552 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.537+0000 7f88cd2c7700 1 --2- 172.21.15.40:0/2295086367 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88b4004e40 0x7f88c812b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:48.553 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.537+0000 7f88cdac8700 1 -- 172.21.15.40:0/2295086367 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f88c80accf0 msgr2=0x7f88c8143260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:48.553 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.537+0000 7f88cdac8700 1 --2- 172.21.15.40:0/2295086367 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f88c80accf0 0x7f88c8143260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.553 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.537+0000 7f88cdac8700 1 -- 172.21.15.40:0/2295086367 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88b4004e40 msgr2=0x7f88c812b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:48.553 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.537+0000 7f88cdac8700 1 --2- 172.21.15.40:0/2295086367 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88b4004e40 0x7f88c812b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.553 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.537+0000 7f88cdac8700 1 -- 172.21.15.40:0/2295086367 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c80adb40 con 0x7f88c80ad170 2020-11-20T13:56:48.554 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.537+0000 7f88cd2c7700 1 --2- 172.21.15.40:0/2295086367 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88b4004e40 0x7f88c812b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:48.554 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.537+0000 7f88cdac8700 1 --2- 172.21.15.40:0/2295086367 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88c80ad170 0x7f88c8143750 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f88b80013e0 tx=0x7f88b8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:48.554 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.537+0000 7f88c67fc700 1 -- 172.21.15.40:0/2295086367 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88b8000e50 con 0x7f88c80ad170 2020-11-20T13:56:48.554 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.537+0000 7f88c67fc700 1 -- 172.21.15.40:0/2295086367 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b8005d30 con 0x7f88c80ad170 2020-11-20T13:56:48.555 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.537+0000 7f88cf52b700 1 -- 172.21.15.40:0/2295086367 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88c80aeb00 con 0x7f88c80ad170 2020-11-20T13:56:48.555 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.537+0000 7f88c67fc700 1 -- 172.21.15.40:0/2295086367 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88b8005ea0 con 0x7f88c80ad170 2020-11-20T13:56:48.555 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.537+0000 7f88cf52b700 1 -- 172.21.15.40:0/2295086367 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88c8151b60 con 0x7f88c80ad170 2020-11-20T13:56:48.555 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.538+0000 7f88cf52b700 1 -- 172.21.15.40:0/2295086367 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88c80adcd0 con 0x7f88c80ad170 2020-11-20T13:56:48.555 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.538+0000 7f88c67fc700 1 -- 172.21.15.40:0/2295086367 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f88b8011470 con 0x7f88c80ad170 2020-11-20T13:56:48.556 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.539+0000 7f88c67fc700 1 --2- 172.21.15.40:0/2295086367 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f88a80513c0 0x7f88a8053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:48.556 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.539+0000 7f88c67fc700 1 -- 172.21.15.40:0/2295086367 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f88b806c830 con 0x7f88c80ad170 2020-11-20T13:56:48.556 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.539+0000 7f88cd2c7700 1 --2- 172.21.15.40:0/2295086367 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f88a80513c0 0x7f88a8053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:48.556 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.539+0000 7f88cd2c7700 1 --2- 172.21.15.40:0/2295086367 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f88a80513c0 0x7f88a8053880 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f88c0003000 tx=0x7f88c0007ea0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:48.556 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.542+0000 7f88c67fc700 1 -- 172.21.15.40:0/2295086367 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f88b80449f0 con 0x7f88c80ad170 2020-11-20T13:56:48.705 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.703+0000 7f88cf52b700 1 -- 172.21.15.40:0/2295086367 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f88c8057050 con 0x7f88c80ad170 2020-11-20T13:56:48.705 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.704+0000 7f88c67fc700 1 -- 172.21.15.40:0/2295086367 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+10 (secure 0 0 0) 0x7f88b8021020 con 0x7f88c80ad170 2020-11-20T13:56:48.705 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2020-11-20T13:56:48.707 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.705+0000 7f88cf52b700 1 -- 172.21.15.40:0/2295086367 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f88a80513c0 msgr2=0x7f88a8053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:48.707 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.705+0000 7f88cf52b700 1 --2- 172.21.15.40:0/2295086367 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f88a80513c0 0x7f88a8053880 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f88c0003000 tx=0x7f88c0007ea0).stop 2020-11-20T13:56:48.708 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.705+0000 7f88cf52b700 1 -- 172.21.15.40:0/2295086367 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88c80ad170 msgr2=0x7f88c8143750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:48.708 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.705+0000 7f88cf52b700 1 --2- 172.21.15.40:0/2295086367 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88c80ad170 0x7f88c8143750 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f88b80013e0 tx=0x7f88b8017000).stop 2020-11-20T13:56:48.708 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.706+0000 7f88cf52b700 1 -- 172.21.15.40:0/2295086367 shutdown_connections 2020-11-20T13:56:48.708 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.706+0000 7f88cf52b700 1 --2- 172.21.15.40:0/2295086367 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f88a80513c0 0x7f88a8053880 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.709 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.706+0000 7f88cf52b700 1 --2- 172.21.15.40:0/2295086367 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88c80ad170 0x7f88c8143750 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.709 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.706+0000 7f88cf52b700 1 --2- 172.21.15.40:0/2295086367 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88b4004e40 0x7f88c812b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.709 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.706+0000 7f88cf52b700 1 --2- 172.21.15.40:0/2295086367 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f88c80accf0 0x7f88c8143260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.709 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.706+0000 7f88cf52b700 1 -- 172.21.15.40:0/2295086367 >> 172.21.15.40:0/2295086367 conn(0x7f88c80ba1d0 msgr2=0x7f88c80b49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:48.709 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.706+0000 7f88cf52b700 1 -- 172.21.15.40:0/2295086367 shutdown_connections 2020-11-20T13:56:48.710 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.706+0000 7f88cf52b700 1 -- 172.21.15.40:0/2295086367 wait complete. 2020-11-20T13:56:48.717 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T13:56:48.717 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-20T13:56:48.832 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.831+0000 7fcb55a9b700 1 Processor -- start 2020-11-20T13:56:48.832 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.831+0000 7fcb55a9b700 1 -- start start 2020-11-20T13:56:48.834 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.831+0000 7fcb55a9b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcb500b6de0 0x7fcb5009b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:48.834 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.831+0000 7fcb55a9b700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fcb500b8ed0 0x7fcb50143720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:48.834 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.831+0000 7fcb55a9b700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb5009bfe0 con 0x7fcb500ae810 2020-11-20T13:56:48.835 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.831+0000 7fcb55a9b700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb5009c280 con 0x7fcb500b6de0 2020-11-20T13:56:48.835 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.831+0000 7fcb55a9b700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb500b71c0 con 0x7fcb500b8ed0 2020-11-20T13:56:48.835 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.831+0000 7fcb4f7fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcb500b6de0 0x7fcb5009b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:48.835 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.832+0000 7fcb4f7fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcb500b6de0 0x7fcb5009b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55428/0 (socket says 172.21.15.40:55428) 2020-11-20T13:56:48.836 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.832+0000 7fcb4f7fe700 1 -- 172.21.15.40:0/2192658427 learned_addr learned my addr 172.21.15.40:0/2192658427 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:48.836 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.832+0000 7fcb4ffff700 1 --2- 172.21.15.40:0/2192658427 >> v2:172.21.15.116:3300/0 conn(0x7fcb500b8ed0 0x7fcb50143720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:48.836 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.832+0000 7fcb4f7fe700 1 -- 172.21.15.40:0/2192658427 >> v2:172.21.15.116:3300/0 conn(0x7fcb500b8ed0 msgr2=0x7fcb50143720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:48.836 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.832+0000 7fcb4f7fe700 1 --2- 172.21.15.40:0/2192658427 >> v2:172.21.15.116:3300/0 conn(0x7fcb500b8ed0 0x7fcb50143720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.836 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.832+0000 7fcb4f7fe700 1 -- 172.21.15.40:0/2192658427 >> v1:172.21.15.116:6789/0 conn(0x7fcb500ae810 legacy=0x7fcb50099050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:48.837 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.832+0000 7fcb4f7fe700 1 -- 172.21.15.40:0/2192658427 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb500af380 con 0x7fcb500b6de0 2020-11-20T13:56:48.837 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.832+0000 7fcb4f7fe700 1 --2- 172.21.15.40:0/2192658427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb500b6de0 0x7fcb5009b330 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7fcb480032b0 tx=0x7fcb48009270).ready entity=mon.0 client_cookie=de74faa9aa18e7d1 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:48.837 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.832+0000 7fcb4e7fc700 1 -- 172.21.15.40:0/2192658427 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb4800bc00 con 0x7fcb500b6de0 2020-11-20T13:56:48.837 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.832+0000 7fcb4e7fc700 1 -- 172.21.15.40:0/2192658427 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4800bd60 con 0x7fcb500b6de0 2020-11-20T13:56:48.837 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.832+0000 7fcb4e7fc700 1 -- 172.21.15.40:0/2192658427 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb48011400 con 0x7fcb500b6de0 2020-11-20T13:56:48.838 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.832+0000 7fcb55a9b700 1 -- 172.21.15.40:0/2192658427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb500b6de0 msgr2=0x7fcb5009b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:48.838 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.832+0000 7fcb55a9b700 1 --2- 172.21.15.40:0/2192658427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb500b6de0 0x7fcb5009b330 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7fcb480032b0 tx=0x7fcb48009270).stop 2020-11-20T13:56:48.838 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.833+0000 7fcb55a9b700 1 -- 172.21.15.40:0/2192658427 shutdown_connections 2020-11-20T13:56:48.838 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.833+0000 7fcb55a9b700 1 --2- 172.21.15.40:0/2192658427 >> v2:172.21.15.116:3300/0 conn(0x7fcb500b8ed0 0x7fcb50143720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.839 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.833+0000 7fcb55a9b700 1 --2- 172.21.15.40:0/2192658427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb500b6de0 0x7fcb5009b330 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.839 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.833+0000 7fcb55a9b700 1 -- 172.21.15.40:0/2192658427 >> 172.21.15.40:0/2192658427 conn(0x7fcb500ba380 msgr2=0x7fcb500b6350 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:48.839 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.833+0000 7fcb55a9b700 1 -- 172.21.15.40:0/2192658427 shutdown_connections 2020-11-20T13:56:48.840 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.833+0000 7fcb55a9b700 1 -- 172.21.15.40:0/2192658427 wait complete. 2020-11-20T13:56:48.843 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.834+0000 7fcb55a9b700 1 Processor -- start 2020-11-20T13:56:48.843 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.834+0000 7fcb55a9b700 1 -- start start 2020-11-20T13:56:48.843 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.834+0000 7fcb55a9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb500ae810 0x7fcb501464c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:48.843 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.834+0000 7fcb55a9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcb500b6de0 0x7fcb500b77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:48.844 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.835+0000 7fcb55a9b700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcb500b8ed0 0x7fcb500b7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:48.844 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.835+0000 7fcb55a9b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb50064cf0 con 0x7fcb500ae810 2020-11-20T13:56:48.844 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.835+0000 7fcb55a9b700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb50064f90 con 0x7fcb500b6de0 2020-11-20T13:56:48.844 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.835+0000 7fcb55a9b700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb50065230 con 0x7fcb500b8ed0 2020-11-20T13:56:48.844 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.835+0000 7fcb4f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb500ae810 0x7fcb501464c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:48.845 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.835+0000 7fcb4f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb500ae810 0x7fcb501464c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55432/0 (socket says 172.21.15.40:55432) 2020-11-20T13:56:48.845 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.835+0000 7fcb4f7fe700 1 -- 172.21.15.40:0/1472699402 learned_addr learned my addr 172.21.15.40:0/1472699402 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:48.845 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.835+0000 7fcb4ffff700 1 --2- 172.21.15.40:0/1472699402 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcb500b8ed0 0x7fcb500b7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:48.845 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.835+0000 7fcb4effd700 1 --2- 172.21.15.40:0/1472699402 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcb500b6de0 0x7fcb500b77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:48.845 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.835+0000 7fcb4f7fe700 1 -- 172.21.15.40:0/1472699402 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcb500b8ed0 msgr2=0x7fcb500b7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:48.846 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.835+0000 7fcb4f7fe700 1 --2- 172.21.15.40:0/1472699402 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcb500b8ed0 0x7fcb500b7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.846 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.835+0000 7fcb4f7fe700 1 -- 172.21.15.40:0/1472699402 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcb500b6de0 msgr2=0x7fcb500b77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:48.846 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.835+0000 7fcb4f7fe700 1 --2- 172.21.15.40:0/1472699402 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcb500b6de0 0x7fcb500b77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.846 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.835+0000 7fcb4f7fe700 1 -- 172.21.15.40:0/1472699402 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb50093c30 con 0x7fcb500ae810 2020-11-20T13:56:48.846 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.835+0000 7fcb4effd700 1 --2- 172.21.15.40:0/1472699402 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcb500b6de0 0x7fcb500b77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:48.847 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.835+0000 7fcb4f7fe700 1 --2- 172.21.15.40:0/1472699402 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb500ae810 0x7fcb501464c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7fcb480032b0 tx=0x7fcb48016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:48.847 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.835+0000 7fcb4cff9700 1 -- 172.21.15.40:0/1472699402 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb480107a0 con 0x7fcb500ae810 2020-11-20T13:56:48.847 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.835+0000 7fcb4cff9700 1 -- 172.21.15.40:0/1472699402 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb48010900 con 0x7fcb500ae810 2020-11-20T13:56:48.847 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.835+0000 7fcb55a9b700 1 -- 172.21.15.40:0/1472699402 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb500b1f30 con 0x7fcb500ae810 2020-11-20T13:56:48.848 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.835+0000 7fcb4cff9700 1 -- 172.21.15.40:0/1472699402 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb48010c60 con 0x7fcb500ae810 2020-11-20T13:56:48.848 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.835+0000 7fcb55a9b700 1 -- 172.21.15.40:0/1472699402 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb50064250 con 0x7fcb500ae810 2020-11-20T13:56:48.848 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.836+0000 7fcb55a9b700 1 -- 172.21.15.40:0/1472699402 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb500653f0 con 0x7fcb500ae810 2020-11-20T13:56:48.848 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.836+0000 7fcb4cff9700 1 -- 172.21.15.40:0/1472699402 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fcb48026030 con 0x7fcb500ae810 2020-11-20T13:56:48.848 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.837+0000 7fcb4cff9700 1 --2- 172.21.15.40:0/1472699402 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fcb300513c0 0x7fcb30053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:48.849 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.837+0000 7fcb4cff9700 1 -- 172.21.15.40:0/1472699402 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fcb4806a250 con 0x7fcb500ae810 2020-11-20T13:56:48.849 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.837+0000 7fcb4effd700 1 --2- 172.21.15.40:0/1472699402 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fcb300513c0 0x7fcb30053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:48.849 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.837+0000 7fcb4effd700 1 --2- 172.21.15.40:0/1472699402 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fcb300513c0 0x7fcb30053880 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fcb4400b000 tx=0x7fcb44009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:48.849 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:48.840+0000 7fcb4cff9700 1 -- 172.21.15.40:0/1472699402 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fcb48042470 con 0x7fcb500ae810 2020-11-20T13:56:49.060 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:49.058+0000 7fcb55a9b700 1 -- 172.21.15.40:0/1472699402 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb50057050 con 0x7fcb500ae810 2020-11-20T13:56:49.060 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:49.059+0000 7fcb4cff9700 1 -- 172.21.15.40:0/1472699402 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+2134 (secure 0 0 0) 0x7fcb48024870 con 0x7fcb500ae810 2020-11-20T13:56:49.060 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:56:49.060 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/2019740031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2019740031},{"type":"v1","addr":"172.21.15.116:6835","nonce":2019740031}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T13:56:25.428178+0000","modified":"2020-11-20T13:56:27.432735+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6833/584131564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":584131564},{"type":"v1","addr":"172.21.15.40:6833","nonce":584131564}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-20T13:56:49.062 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:49.060+0000 7fcb55a9b700 1 -- 172.21.15.40:0/1472699402 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fcb300513c0 msgr2=0x7fcb30053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:49.062 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:49.060+0000 7fcb55a9b700 1 --2- 172.21.15.40:0/1472699402 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fcb300513c0 0x7fcb30053880 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fcb4400b000 tx=0x7fcb44009040).stop 2020-11-20T13:56:49.063 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:49.060+0000 7fcb55a9b700 1 -- 172.21.15.40:0/1472699402 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb500ae810 msgr2=0x7fcb501464c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:49.063 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:49.060+0000 7fcb55a9b700 1 --2- 172.21.15.40:0/1472699402 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb500ae810 0x7fcb501464c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7fcb480032b0 tx=0x7fcb48016000).stop 2020-11-20T13:56:49.063 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:49.060+0000 7fcb55a9b700 1 -- 172.21.15.40:0/1472699402 shutdown_connections 2020-11-20T13:56:49.063 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:49.060+0000 7fcb55a9b700 1 --2- 172.21.15.40:0/1472699402 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fcb300513c0 0x7fcb30053880 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:49.063 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:49.060+0000 7fcb55a9b700 1 --2- 172.21.15.40:0/1472699402 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcb500b8ed0 0x7fcb500b7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:49.064 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:49.060+0000 7fcb55a9b700 1 --2- 172.21.15.40:0/1472699402 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb500ae810 0x7fcb501464c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:49.064 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:49.060+0000 7fcb55a9b700 1 --2- 172.21.15.40:0/1472699402 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcb500b6de0 0x7fcb500b77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:49.064 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:49.060+0000 7fcb55a9b700 1 -- 172.21.15.40:0/1472699402 >> 172.21.15.40:0/1472699402 conn(0x7fcb500ba380 msgr2=0x7fcb500b4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:49.064 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:49.061+0000 7fcb55a9b700 1 -- 172.21.15.40:0/1472699402 shutdown_connections 2020-11-20T13:56:49.065 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:49.061+0000 7fcb55a9b700 1 -- 172.21.15.40:0/1472699402 wait complete. 2020-11-20T13:56:49.065 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 7 2020-11-20T13:56:49.274 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T13:56:25.428178+0000', 'modified': '2020-11-20T13:56:27.432735+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4263}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4263': {'gid': 4263, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.40:6833/584131564', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6832', 'nonce': 584131564}, {'type': 'v1', 'addr': '172.21.15.40:6833', 'nonce': 584131564}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T13:56:49.274 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T13:56:49.275 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T13:56:49.412 INFO:teuthology.orchestra.run.smithi040.stdout:{"cluster_fsid":"da1ed737-5174-430f-a3c9-56f6bbfad13c","whoami":0,"id":4263,"want_state":"up:active","state":"up:active","rank_uptime":23.974244415000001,"mdsmap_epoch":7,"osdmap_epoch":24,"osdmap_epoch_barrier":24,"uptime":30.288788615000001} 2020-11-20T13:56:49.412 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "da1ed737-5174-430f-a3c9-56f6bbfad13c", "id": 4263, "mdsmap_epoch": 7, "osdmap_epoch": 24, "osdmap_epoch_barrier": 24, "rank_uptime": 23.974244415, "state": "up:active", "uptime": 30.288788615, "want_state": "up:active", "whoami": 0 } 2020-11-20T13:56:49.413 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-11-20T13:56:49.413 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2020-11-20T13:56:49.413 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.3 at ubuntu@smithi116.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.3... 2020-11-20T13:56:49.414 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:49.414 INFO:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.3) 2020-11-20T13:56:49.432 INFO:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2020-11-20T13:56:49.505 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-20T13:56:49.505 INFO:teuthology.orchestra.run.smithi116:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-20T13:56:49.576 INFO:teuthology.orchestra.run.smithi116.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-11-20T13:56:49.577 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-20T13:56:49.577 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:49.578 INFO:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2020-11-20T13:56:49.634 INFO:teuthology.orchestra.run.smithi116.stdout:47 2020-11-20T13:56:49.634 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [47] 2020-11-20T13:56:49.635 INFO:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.3 /home/ubuntu/cephtest/mnt.3) 2020-11-20T13:56:49.676 INFO:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2020-11-20T13:56:49.708 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-20T13:56:49.708 INFO:teuthology.orchestra.run.smithi116:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-20T13:56:49.757 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi116.stderr:2020-11-20T13:56:49.757+0000 7f3899a001c0 -1 init, newargv = 0x55c4b84cd5f0 newargc=9 2020-11-20T13:56:49.758 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi116.stderr:ceph-fuse[24087]: starting ceph client 2020-11-20T13:56:49.762 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi116.stderr:ceph-fuse[24087]: starting fuse 2020-11-20T13:56:49.779 INFO:teuthology.orchestra.run.smithi116.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-11-20T13:56:49.780 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-20T13:56:49.781 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:49.781 INFO:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2020-11-20T13:56:49.835 INFO:teuthology.orchestra.run.smithi116.stdout:47 2020-11-20T13:56:49.835 INFO:teuthology.orchestra.run.smithi116.stdout:48 2020-11-20T13:56:49.835 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47, 48] 2020-11-20T13:56:49.836 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:49.836 INFO:teuthology.orchestra.run.smithi116:> sudo python3 -c ' 2020-11-20T13:56:49.836 INFO:teuthology.orchestra.run.smithi116:> import glob 2020-11-20T13:56:49.837 INFO:teuthology.orchestra.run.smithi116:> import re 2020-11-20T13:56:49.837 INFO:teuthology.orchestra.run.smithi116:> import os 2020-11-20T13:56:49.837 INFO:teuthology.orchestra.run.smithi116:> import subprocess 2020-11-20T13:56:49.838 INFO:teuthology.orchestra.run.smithi116:> 2020-11-20T13:56:49.838 INFO:teuthology.orchestra.run.smithi116:> def find_socket(client_name): 2020-11-20T13:56:49.838 INFO:teuthology.orchestra.run.smithi116:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2020-11-20T13:56:49.838 INFO:teuthology.orchestra.run.smithi116:> files = glob.glob(asok_path) 2020-11-20T13:56:49.839 INFO:teuthology.orchestra.run.smithi116:> 2020-11-20T13:56:49.839 INFO:teuthology.orchestra.run.smithi116:> # Given a non-glob path, it better be there 2020-11-20T13:56:49.839 INFO:teuthology.orchestra.run.smithi116:> if "*" not in asok_path: 2020-11-20T13:56:49.840 INFO:teuthology.orchestra.run.smithi116:> assert(len(files) == 1) 2020-11-20T13:56:49.840 INFO:teuthology.orchestra.run.smithi116:> return files[0] 2020-11-20T13:56:49.840 INFO:teuthology.orchestra.run.smithi116:> 2020-11-20T13:56:49.840 INFO:teuthology.orchestra.run.smithi116:> for f in files: 2020-11-20T13:56:49.841 INFO:teuthology.orchestra.run.smithi116:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-11-20T13:56:49.841 INFO:teuthology.orchestra.run.smithi116:> if os.path.exists("/proc/{0}".format(pid)): 2020-11-20T13:56:49.841 INFO:teuthology.orchestra.run.smithi116:> return f 2020-11-20T13:56:49.842 INFO:teuthology.orchestra.run.smithi116:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-11-20T13:56:49.842 INFO:teuthology.orchestra.run.smithi116:> 2020-11-20T13:56:49.842 INFO:teuthology.orchestra.run.smithi116:> print(find_socket("client.3")) 2020-11-20T13:56:49.842 INFO:teuthology.orchestra.run.smithi116:> ' 2020-11-20T13:56:49.926 INFO:teuthology.orchestra.run.smithi116.stdout:/var/run/ceph/ceph-client.3.24087.asok 2020-11-20T13:56:49.930 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.24087.asok 2020-11-20T13:56:49.931 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:49.931 INFO:teuthology.orchestra.run.smithi116:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.24087.asok status 2020-11-20T13:56:50.034 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2020-11-20T13:56:50.034 INFO:teuthology.orchestra.run.smithi116.stdout: "metadata": { 2020-11-20T13:56:50.034 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_sha1": "a7b62340ad27b26642bb54ec916a97c7e1f07a34", 2020-11-20T13:56:50.035 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_version": "ceph version 15.2.5-588-ga7b62340 (a7b62340ad27b26642bb54ec916a97c7e1f07a34) octopus (stable)", 2020-11-20T13:56:50.035 INFO:teuthology.orchestra.run.smithi116.stdout: "entity_id": "3", 2020-11-20T13:56:50.035 INFO:teuthology.orchestra.run.smithi116.stdout: "hostname": "smithi116", 2020-11-20T13:56:50.035 INFO:teuthology.orchestra.run.smithi116.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2020-11-20T13:56:50.035 INFO:teuthology.orchestra.run.smithi116.stdout: "pid": "24087", 2020-11-20T13:56:50.036 INFO:teuthology.orchestra.run.smithi116.stdout: "root": "/" 2020-11-20T13:56:50.036 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T13:56:50.036 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_count": 0, 2020-11-20T13:56:50.036 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_pinned_count": 0, 2020-11-20T13:56:50.036 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 4693, 2020-11-20T13:56:50.037 INFO:teuthology.orchestra.run.smithi116.stdout: "inst": { 2020-11-20T13:56:50.037 INFO:teuthology.orchestra.run.smithi116.stdout: "name": { 2020-11-20T13:56:50.037 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "client", 2020-11-20T13:56:50.037 INFO:teuthology.orchestra.run.smithi116.stdout: "num": 4693 2020-11-20T13:56:50.037 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T13:56:50.038 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2020-11-20T13:56:50.038 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T13:56:50.038 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:0", 2020-11-20T13:56:50.038 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 2842006708 2020-11-20T13:56:50.038 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T13:56:50.039 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T13:56:50.039 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2020-11-20T13:56:50.039 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T13:56:50.040 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:0", 2020-11-20T13:56:50.040 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 2842006708 2020-11-20T13:56:50.040 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T13:56:50.040 INFO:teuthology.orchestra.run.smithi116.stdout: "inst_str": "client.4693 172.21.15.116:0/2842006708", 2020-11-20T13:56:50.040 INFO:teuthology.orchestra.run.smithi116.stdout: "addr_str": "172.21.15.116:0/2842006708", 2020-11-20T13:56:50.041 INFO:teuthology.orchestra.run.smithi116.stdout: "inode_count": 1, 2020-11-20T13:56:50.041 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_epoch": 7, 2020-11-20T13:56:50.041 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch": 24, 2020-11-20T13:56:50.041 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch_barrier": 0, 2020-11-20T13:56:50.041 INFO:teuthology.orchestra.run.smithi116.stdout: "blacklisted": false 2020-11-20T13:56:50.042 INFO:teuthology.orchestra.run.smithi116.stdout:} 2020-11-20T13:56:50.043 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:50.043 INFO:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-20T13:56:50.043 INFO:teuthology.orchestra.run.smithi040:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-20T13:56:50.061 INFO:teuthology.orchestra.run.smithi040.stdout:fuseblk 2020-11-20T13:56:50.061 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-20T13:56:50.061 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:50.062 INFO:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-11-20T13:56:50.145 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:50.145 INFO:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-20T13:56:50.146 INFO:teuthology.orchestra.run.smithi040:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-20T13:56:50.204 INFO:teuthology.orchestra.run.smithi040.stdout:fuseblk 2020-11-20T13:56:50.204 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-11-20T13:56:50.204 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:50.205 INFO:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2020-11-20T13:56:50.283 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:50.284 INFO:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-20T13:56:50.284 INFO:teuthology.orchestra.run.smithi116:> ' -- /home/ubuntu/cephtest/mnt.2) 2020-11-20T13:56:50.300 INFO:teuthology.orchestra.run.smithi116.stdout:fuseblk 2020-11-20T13:56:50.300 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2020-11-20T13:56:50.301 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:50.301 INFO:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.2) 2020-11-20T13:56:50.373 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:50.374 INFO:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-20T13:56:50.374 INFO:teuthology.orchestra.run.smithi116:> ' -- /home/ubuntu/cephtest/mnt.3) 2020-11-20T13:56:50.428 INFO:teuthology.orchestra.run.smithi116.stdout:fuseblk 2020-11-20T13:56:50.428 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2020-11-20T13:56:50.429 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:50.429 INFO:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.3) 2020-11-20T13:56:50.499 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2020-11-20T13:56:50.522 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-20T13:56:50.649 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.648+0000 7fa9f092e700 1 Processor -- start 2020-11-20T13:56:50.650 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.648+0000 7fa9f092e700 1 -- start start 2020-11-20T13:56:50.652 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.649+0000 7fa9f092e700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa9ec09a280 0x7fa9ec0b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:50.652 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.649+0000 7fa9f092e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa9ec09bc40 0x7fa9ec09c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:50.652 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.649+0000 7fa9f092e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9ec0b6be0 con 0x7fa9ec0b88c0 2020-11-20T13:56:50.652 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.649+0000 7fa9f092e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9ec099000 con 0x7fa9ec09bc40 2020-11-20T13:56:50.653 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.649+0000 7fa9f092e700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ec0b4250 con 0x7fa9ec09a280 2020-11-20T13:56:50.653 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.649+0000 7fa9ead9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa9ec09bc40 0x7fa9ec09c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:50.653 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.650+0000 7fa9ead9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa9ec09bc40 0x7fa9ec09c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55440/0 (socket says 172.21.15.40:55440) 2020-11-20T13:56:50.653 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.650+0000 7fa9ead9d700 1 -- 172.21.15.40:0/3706664755 learned_addr learned my addr 172.21.15.40:0/3706664755 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:50.653 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.650+0000 7fa9e9d9b700 1 --2- 172.21.15.40:0/3706664755 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa9ec09a280 0x7fa9ec0b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:50.654 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.650+0000 7fa9ead9d700 1 -- 172.21.15.40:0/3706664755 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa9ec09a280 msgr2=0x7fa9ec0b8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:50.654 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.650+0000 7fa9ead9d700 1 --2- 172.21.15.40:0/3706664755 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa9ec09a280 0x7fa9ec0b8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:50.654 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.650+0000 7fa9ead9d700 1 -- 172.21.15.40:0/3706664755 >> v1:172.21.15.116:6789/0 conn(0x7fa9ec0b88c0 legacy=0x7fa9ec09b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:50.655 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.650+0000 7fa9ead9d700 1 -- 172.21.15.40:0/3706664755 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9ec0b4390 con 0x7fa9ec09bc40 2020-11-20T13:56:50.655 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.650+0000 7fa9ead9d700 1 --2- 172.21.15.40:0/3706664755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9ec09bc40 0x7fa9ec09c0a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7fa9e4002f30 tx=0x7fa9e4006e80).ready entity=mon.0 client_cookie=a13ef27c524a3e6e server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:50.655 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.650+0000 7fa9e959a700 1 -- 172.21.15.40:0/3706664755 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9e40097f0 con 0x7fa9ec09bc40 2020-11-20T13:56:50.655 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.650+0000 7fa9e959a700 1 -- 172.21.15.40:0/3706664755 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e4009950 con 0x7fa9ec09bc40 2020-11-20T13:56:50.656 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.650+0000 7fa9e959a700 1 -- 172.21.15.40:0/3706664755 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9e4009cb0 con 0x7fa9ec09bc40 2020-11-20T13:56:50.656 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.650+0000 7fa9f092e700 1 -- 172.21.15.40:0/3706664755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9ec09bc40 msgr2=0x7fa9ec09c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:50.656 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.650+0000 7fa9f092e700 1 --2- 172.21.15.40:0/3706664755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9ec09bc40 0x7fa9ec09c0a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7fa9e4002f30 tx=0x7fa9e4006e80).stop 2020-11-20T13:56:50.656 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.651+0000 7fa9f092e700 1 -- 172.21.15.40:0/3706664755 shutdown_connections 2020-11-20T13:56:50.656 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.651+0000 7fa9f092e700 1 --2- 172.21.15.40:0/3706664755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9ec09bc40 0x7fa9ec09c0a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:50.657 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.651+0000 7fa9f092e700 1 --2- 172.21.15.40:0/3706664755 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa9ec09a280 0x7fa9ec0b8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:50.657 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.651+0000 7fa9f092e700 1 -- 172.21.15.40:0/3706664755 >> 172.21.15.40:0/3706664755 conn(0x7fa9ec0ba380 msgr2=0x7fa9ec0b6350 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:50.657 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.651+0000 7fa9f092e700 1 -- 172.21.15.40:0/3706664755 shutdown_connections 2020-11-20T13:56:50.657 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.651+0000 7fa9f092e700 1 -- 172.21.15.40:0/3706664755 wait complete. 2020-11-20T13:56:50.660 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.652+0000 7fa9f092e700 1 Processor -- start 2020-11-20T13:56:50.661 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.652+0000 7fa9f092e700 1 -- start start 2020-11-20T13:56:50.661 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.652+0000 7fa9f092e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9ec09a280 0x7fa9ec148a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:50.661 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.652+0000 7fa9f092e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9ec09bc40 0x7fa9ec0b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:50.661 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.653+0000 7fa9f092e700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa9ec0b88c0 0x7fa9ec146420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:50.662 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.653+0000 7fa9f092e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ec141720 con 0x7fa9ec09a280 2020-11-20T13:56:50.662 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.653+0000 7fa9f092e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ec0b15b0 con 0x7fa9ec09bc40 2020-11-20T13:56:50.662 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.653+0000 7fa9f092e700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ec0b1850 con 0x7fa9ec0b88c0 2020-11-20T13:56:50.662 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.653+0000 7fa9ea59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9ec09a280 0x7fa9ec148a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:50.662 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.653+0000 7fa9ea59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9ec09a280 0x7fa9ec148a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55446/0 (socket says 172.21.15.40:55446) 2020-11-20T13:56:50.663 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.653+0000 7fa9ea59c700 1 -- 172.21.15.40:0/4208076369 learned_addr learned my addr 172.21.15.40:0/4208076369 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:50.663 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.653+0000 7fa9ead9d700 1 --2- 172.21.15.40:0/4208076369 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa9ec0b88c0 0x7fa9ec146420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:50.663 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.653+0000 7fa9e9d9b700 1 --2- 172.21.15.40:0/4208076369 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9ec09bc40 0x7fa9ec0b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:50.663 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.653+0000 7fa9ea59c700 1 -- 172.21.15.40:0/4208076369 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa9ec0b88c0 msgr2=0x7fa9ec146420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:50.663 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.653+0000 7fa9ea59c700 1 --2- 172.21.15.40:0/4208076369 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa9ec0b88c0 0x7fa9ec146420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:50.664 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.653+0000 7fa9ea59c700 1 -- 172.21.15.40:0/4208076369 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9ec09bc40 msgr2=0x7fa9ec0b32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:50.664 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.653+0000 7fa9ea59c700 1 --2- 172.21.15.40:0/4208076369 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9ec09bc40 0x7fa9ec0b32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:50.664 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.653+0000 7fa9ea59c700 1 -- 172.21.15.40:0/4208076369 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9ec0b1af0 con 0x7fa9ec09a280 2020-11-20T13:56:50.664 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.653+0000 7fa9e9d9b700 1 --2- 172.21.15.40:0/4208076369 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9ec09bc40 0x7fa9ec0b32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:50.664 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.653+0000 7fa9ea59c700 1 --2- 172.21.15.40:0/4208076369 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9ec09a280 0x7fa9ec148a70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7fa9ec0627d0 tx=0x7fa9d4007b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:50.665 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.653+0000 7fa9e37fe700 1 -- 172.21.15.40:0/4208076369 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9d4007030 con 0x7fa9ec09a280 2020-11-20T13:56:50.665 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.653+0000 7fa9e37fe700 1 -- 172.21.15.40:0/4208076369 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d4007190 con 0x7fa9ec09a280 2020-11-20T13:56:50.665 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.653+0000 7fa9f092e700 1 -- 172.21.15.40:0/4208076369 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9ec146ec0 con 0x7fa9ec09a280 2020-11-20T13:56:50.665 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.653+0000 7fa9e37fe700 1 -- 172.21.15.40:0/4208076369 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9d40064e0 con 0x7fa9ec09a280 2020-11-20T13:56:50.665 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.653+0000 7fa9f092e700 1 -- 172.21.15.40:0/4208076369 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9ec0b3860 con 0x7fa9ec09a280 2020-11-20T13:56:50.666 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.655+0000 7fa9f092e700 1 -- 172.21.15.40:0/4208076369 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9ec057050 con 0x7fa9ec09a280 2020-11-20T13:56:50.666 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.655+0000 7fa9e37fe700 1 -- 172.21.15.40:0/4208076369 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fa9d4010070 con 0x7fa9ec09a280 2020-11-20T13:56:50.666 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.655+0000 7fa9e37fe700 1 --2- 172.21.15.40:0/4208076369 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa9d0051290 0x7fa9d0053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:50.666 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.655+0000 7fa9e37fe700 1 -- 172.21.15.40:0/4208076369 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fa9d4068920 con 0x7fa9ec09a280 2020-11-20T13:56:50.666 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.655+0000 7fa9e9d9b700 1 --2- 172.21.15.40:0/4208076369 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa9d0051290 0x7fa9d0053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:50.667 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.655+0000 7fa9e9d9b700 1 --2- 172.21.15.40:0/4208076369 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa9d0051290 0x7fa9d0053750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fa9dc008000 tx=0x7fa9dc008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:50.667 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.659+0000 7fa9e37fe700 1 -- 172.21.15.40:0/4208076369 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fa9d4040bf0 con 0x7fa9ec09a280 2020-11-20T13:56:50.822 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.821+0000 7fa9f092e700 1 -- 172.21.15.40:0/4208076369 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9ec0b9690 con 0x7fa9ec09a280 2020-11-20T13:56:50.823 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.822+0000 7fa9e37fe700 1 -- 172.21.15.40:0/4208076369 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16347 (secure 0 0 0) 0x7fa9d4044210 con 0x7fa9ec09a280 2020-11-20T13:56:50.823 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:56:50.823 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":24,"fsid":"da1ed737-5174-430f-a3c9-56f6bbfad13c","created":"2020-11-20T13:56:04.990651+0000","modified":"2020-11-20T13:56:25.428032+0000","last_up_change":"2020-11-20T13:56:13.893026+0000","last_in_change":"2020-11-20T13:56:13.893026+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T13:56:08.330017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T13:56:17.939684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T13:56:20.563740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T13:56:23.062623+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9ea4c667-8938-46d6-a5ca-092d55c1f79e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":22825},{"type":"v1","addr":"172.21.15.40:6818","nonce":22825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":22825},{"type":"v1","addr":"172.21.15.40:6822","nonce":22825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":22825},{"type":"v1","addr":"172.21.15.40:6830","nonce":22825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":22825},{"type":"v1","addr":"172.21.15.40:6826","nonce":22825}]},"public_addr":"172.21.15.40:6818/22825","cluster_addr":"172.21.15.40:6822/22825","heartbeat_back_addr":"172.21.15.40:6830/22825","heartbeat_front_addr":"172.21.15.40:6826/22825","state":["exists","up"]},{"osd":1,"uuid":"515708d9-5087-4e55-8d14-cdea1c00eacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":22823},{"type":"v1","addr":"172.21.15.40:6801","nonce":22823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":22823},{"type":"v1","addr":"172.21.15.40:6803","nonce":22823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":22823},{"type":"v1","addr":"172.21.15.40:6807","nonce":22823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":22823},{"type":"v1","addr":"172.21.15.40:6805","nonce":22823}]},"public_addr":"172.21.15.40:6801/22823","cluster_addr":"172.21.15.40:6803/22823","heartbeat_back_addr":"172.21.15.40:6807/22823","heartbeat_front_addr":"172.21.15.40:6805/22823","state":["exists","up"]},{"osd":2,"uuid":"4d86bca7-743e-4983-b35f-36676c405b36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":22824},{"type":"v1","addr":"172.21.15.40:6809","nonce":22824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":22824},{"type":"v1","addr":"172.21.15.40:6811","nonce":22824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":22824},{"type":"v1","addr":"172.21.15.40:6815","nonce":22824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":22824},{"type":"v1","addr":"172.21.15.40:6813","nonce":22824}]},"public_addr":"172.21.15.40:6809/22824","cluster_addr":"172.21.15.40:6811/22824","heartbeat_back_addr":"172.21.15.40:6815/22824","heartbeat_front_addr":"172.21.15.40:6813/22824","state":["exists","up"]},{"osd":3,"uuid":"f4dc726b-6b34-4e8b-8eeb-87060b58002d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":22826},{"type":"v1","addr":"172.21.15.40:6819","nonce":22826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":22826},{"type":"v1","addr":"172.21.15.40:6823","nonce":22826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":22826},{"type":"v1","addr":"172.21.15.40:6831","nonce":22826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":22826},{"type":"v1","addr":"172.21.15.40:6827","nonce":22826}]},"public_addr":"172.21.15.40:6819/22826","cluster_addr":"172.21.15.40:6823/22826","heartbeat_back_addr":"172.21.15.40:6831/22826","heartbeat_front_addr":"172.21.15.40:6827/22826","state":["exists","up"]},{"osd":4,"uuid":"25a6d53b-d1f0-4498-b7d5-ffa38661d403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":23181},{"type":"v1","addr":"172.21.15.116:6819","nonce":23181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":23181},{"type":"v1","addr":"172.21.15.116:6821","nonce":23181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":23181},{"type":"v1","addr":"172.21.15.116:6825","nonce":23181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":23181},{"type":"v1","addr":"172.21.15.116:6823","nonce":23181}]},"public_addr":"172.21.15.116:6819/23181","cluster_addr":"172.21.15.116:6821/23181","heartbeat_back_addr":"172.21.15.116:6825/23181","heartbeat_front_addr":"172.21.15.116:6823/23181","state":["exists","up"]},{"osd":5,"uuid":"7d7a5af8-229b-4351-9a21-a3d509cd4635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":23171},{"type":"v1","addr":"172.21.15.116:6803","nonce":23171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":23171},{"type":"v1","addr":"172.21.15.116:6805","nonce":23171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":23171},{"type":"v1","addr":"172.21.15.116:6809","nonce":23171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":23171},{"type":"v1","addr":"172.21.15.116:6807","nonce":23171}]},"public_addr":"172.21.15.116:6803/23171","cluster_addr":"172.21.15.116:6805/23171","heartbeat_back_addr":"172.21.15.116:6809/23171","heartbeat_front_addr":"172.21.15.116:6807/23171","state":["exists","up"]},{"osd":6,"uuid":"8247d248-a140-4e36-9796-9847a97df58d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":23172},{"type":"v1","addr":"172.21.15.116:6811","nonce":23172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":23172},{"type":"v1","addr":"172.21.15.116:6813","nonce":23172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":23172},{"type":"v1","addr":"172.21.15.116:6817","nonce":23172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":23172},{"type":"v1","addr":"172.21.15.116:6815","nonce":23172}]},"public_addr":"172.21.15.116:6811/23172","cluster_addr":"172.21.15.116:6813/23172","heartbeat_back_addr":"172.21.15.116:6817/23172","heartbeat_front_addr":"172.21.15.116:6815/23172","state":["exists","up"]},{"osd":7,"uuid":"472803c4-70b8-499d-b18a-71aaeec6aab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":23196},{"type":"v1","addr":"172.21.15.116:6827","nonce":23196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":23196},{"type":"v1","addr":"172.21.15.116:6829","nonce":23196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":23196},{"type":"v1","addr":"172.21.15.116:6833","nonce":23196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":23196},{"type":"v1","addr":"172.21.15.116:6831","nonce":23196}]},"public_addr":"172.21.15.116:6827/23196","cluster_addr":"172.21.15.116:6829/23196","heartbeat_back_addr":"172.21.15.116:6833/23196","heartbeat_front_addr":"172.21.15.116:6831/23196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.029357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:11.989797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.091097+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.014766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.070860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.071864+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.063400+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.109028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-20T13:56:50.825 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.823+0000 7fa9f092e700 1 -- 172.21.15.40:0/4208076369 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa9d0051290 msgr2=0x7fa9d0053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:50.825 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.823+0000 7fa9f092e700 1 --2- 172.21.15.40:0/4208076369 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa9d0051290 0x7fa9d0053750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fa9dc008000 tx=0x7fa9dc008040).stop 2020-11-20T13:56:50.825 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.823+0000 7fa9f092e700 1 -- 172.21.15.40:0/4208076369 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9ec09a280 msgr2=0x7fa9ec148a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:50.825 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.823+0000 7fa9f092e700 1 --2- 172.21.15.40:0/4208076369 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9ec09a280 0x7fa9ec148a70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7fa9ec0627d0 tx=0x7fa9d4007b50).stop 2020-11-20T13:56:50.826 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.823+0000 7fa9f092e700 1 -- 172.21.15.40:0/4208076369 shutdown_connections 2020-11-20T13:56:50.826 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.823+0000 7fa9f092e700 1 --2- 172.21.15.40:0/4208076369 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fa9d0051290 0x7fa9d0053750 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:50.826 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.823+0000 7fa9f092e700 1 --2- 172.21.15.40:0/4208076369 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa9ec0b88c0 0x7fa9ec146420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:50.826 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.823+0000 7fa9f092e700 1 --2- 172.21.15.40:0/4208076369 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9ec09a280 0x7fa9ec148a70 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:50.827 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.823+0000 7fa9f092e700 1 --2- 172.21.15.40:0/4208076369 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9ec09bc40 0x7fa9ec0b32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:50.827 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.823+0000 7fa9f092e700 1 -- 172.21.15.40:0/4208076369 >> 172.21.15.40:0/4208076369 conn(0x7fa9ec0ba380 msgr2=0x7fa9ec142ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:50.827 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.823+0000 7fa9f092e700 1 -- 172.21.15.40:0/4208076369 shutdown_connections 2020-11-20T13:56:50.827 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.823+0000 7fa9f092e700 1 -- 172.21.15.40:0/4208076369 wait complete. 2020-11-20T13:56:50.836 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T13:56:08.330017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T13:56:17.939684+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T13:56:20.563740+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T13:56:23.062623+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T13:56:50.836 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-20T13:56:50.953 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.952+0000 7f802bfb6700 1 Processor -- start 2020-11-20T13:56:50.954 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.952+0000 7f802bfb6700 1 -- start start 2020-11-20T13:56:50.956 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.953+0000 7f802bfb6700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f802413f400 0x7f80240b3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:50.956 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.953+0000 7f802bfb6700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f80240b6900 0x7f8024145290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:50.956 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.953+0000 7f802bfb6700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80241489e0 con 0x7f802405fa50 2020-11-20T13:56:50.956 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.953+0000 7f802bfb6700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8024148c80 con 0x7f802413f400 2020-11-20T13:56:50.956 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.953+0000 7f802bfb6700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f802405f5c0 con 0x7f80240b6900 2020-11-20T13:56:50.957 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.953+0000 7f8029d52700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f802413f400 0x7f80240b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:50.957 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.953+0000 7f8029551700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f802405fa50 0x7f8024149120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46050/0 (socket says 172.21.15.40:46050) 2020-11-20T13:56:50.957 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.953+0000 7f8029551700 1 -- 172.21.15.40:0/3268734784 learned_addr learned my addr 172.21.15.40:0/3268734784 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:50.957 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.953+0000 7f802a553700 1 --2- 172.21.15.40:0/3268734784 >> v2:172.21.15.116:3300/0 conn(0x7f80240b6900 0x7f8024145290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:50.959 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.953+0000 7f8029d52700 1 -- 172.21.15.40:0/3268734784 >> v2:172.21.15.116:3300/0 conn(0x7f80240b6900 msgr2=0x7f8024145290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:50.959 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.953+0000 7f8029d52700 1 --2- 172.21.15.40:0/3268734784 >> v2:172.21.15.116:3300/0 conn(0x7f80240b6900 0x7f8024145290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:50.959 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.953+0000 7f8029d52700 1 -- 172.21.15.40:0/3268734784 >> v1:172.21.15.40:6789/0 conn(0x7f802405fa50 legacy=0x7f8024149120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:50.959 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.953+0000 7f8029d52700 1 -- 172.21.15.40:0/3268734784 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80240b6db0 con 0x7f802413f400 2020-11-20T13:56:50.959 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.953+0000 7f8028d50700 1 -- 172.21.15.40:0/3268734784 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2949915444 0 0) 0x7f80241489e0 con 0x7f802405fa50 2020-11-20T13:56:50.960 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.954+0000 7f8029d52700 1 --2- 172.21.15.40:0/3268734784 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f802413f400 0x7f80240b3730 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f8014002f30 tx=0x7f8014006d60).ready entity=mon.0 client_cookie=e95ebbd56f844dbb server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:50.960 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.954+0000 7f8028d50700 1 -- 172.21.15.40:0/3268734784 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8014009600 con 0x7f802413f400 2020-11-20T13:56:50.960 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.954+0000 7f8028d50700 1 -- 172.21.15.40:0/3268734784 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8014009760 con 0x7f802413f400 2020-11-20T13:56:50.960 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.954+0000 7f8028d50700 1 -- 172.21.15.40:0/3268734784 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f80140098d0 con 0x7f802413f400 2020-11-20T13:56:50.961 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.954+0000 7f802bfb6700 1 -- 172.21.15.40:0/3268734784 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f802413f400 msgr2=0x7f80240b3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:50.961 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.954+0000 7f802bfb6700 1 --2- 172.21.15.40:0/3268734784 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f802413f400 0x7f80240b3730 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f8014002f30 tx=0x7f8014006d60).stop 2020-11-20T13:56:50.961 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.954+0000 7f802bfb6700 1 -- 172.21.15.40:0/3268734784 shutdown_connections 2020-11-20T13:56:50.961 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.954+0000 7f802bfb6700 1 --2- 172.21.15.40:0/3268734784 >> v2:172.21.15.116:3300/0 conn(0x7f80240b6900 0x7f8024145290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:50.961 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.954+0000 7f802bfb6700 1 --2- 172.21.15.40:0/3268734784 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f802413f400 0x7f80240b3730 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:50.962 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.954+0000 7f802bfb6700 1 -- 172.21.15.40:0/3268734784 >> 172.21.15.40:0/3268734784 conn(0x7f80240ba1d0 msgr2=0x7f80240b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:50.964 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.954+0000 7f802bfb6700 1 -- 172.21.15.40:0/3268734784 shutdown_connections 2020-11-20T13:56:50.964 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.954+0000 7f802bfb6700 1 -- 172.21.15.40:0/3268734784 wait complete. 2020-11-20T13:56:50.966 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.955+0000 7f802bfb6700 1 Processor -- start 2020-11-20T13:56:50.967 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.955+0000 7f802bfb6700 1 -- start start 2020-11-20T13:56:50.967 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.956+0000 7f802bfb6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f802405fa50 0x7f8024067820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:50.968 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.956+0000 7f802bfb6700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f80240b6900 0x7f80240636e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:50.968 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.956+0000 7f802bfb6700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f802413f400 0x7f802413e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:50.968 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.956+0000 7f802bfb6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80240af910 con 0x7f802405fa50 2020-11-20T13:56:50.968 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.956+0000 7f802bfb6700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8024063e30 con 0x7f80240b6900 2020-11-20T13:56:50.968 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.956+0000 7f802bfb6700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8024063bd0 con 0x7f802413f400 2020-11-20T13:56:50.969 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.956+0000 7f8029d52700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f802405fa50 0x7f8024067820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:50.969 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.956+0000 7f8029d52700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f802405fa50 0x7f8024067820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55460/0 (socket says 172.21.15.40:55460) 2020-11-20T13:56:50.969 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.956+0000 7f8029d52700 1 -- 172.21.15.40:0/2840635113 learned_addr learned my addr 172.21.15.40:0/2840635113 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:50.969 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.956+0000 7f8029551700 1 --2- 172.21.15.40:0/2840635113 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f80240b6900 0x7f80240636e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:50.970 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.956+0000 7f802a553700 1 --2- 172.21.15.40:0/2840635113 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f802413f400 0x7f802413e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:50.970 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.956+0000 7f8029d52700 1 -- 172.21.15.40:0/2840635113 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f802413f400 msgr2=0x7f802413e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:50.970 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.956+0000 7f8029d52700 1 --2- 172.21.15.40:0/2840635113 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f802413f400 0x7f802413e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:50.970 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.956+0000 7f8029d52700 1 -- 172.21.15.40:0/2840635113 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f80240b6900 msgr2=0x7f80240636e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:50.970 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.956+0000 7f8029d52700 1 --2- 172.21.15.40:0/2840635113 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f80240b6900 0x7f80240636e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:50.971 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.956+0000 7f8029d52700 1 -- 172.21.15.40:0/2840635113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80240af1d0 con 0x7f802405fa50 2020-11-20T13:56:50.971 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.956+0000 7f8029d52700 1 --2- 172.21.15.40:0/2840635113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f802405fa50 0x7f8024067820 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f8014002f30 tx=0x7f8014016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:50.971 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.957+0000 7f8012ffd700 1 -- 172.21.15.40:0/2840635113 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8014011c10 con 0x7f802405fa50 2020-11-20T13:56:50.971 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.957+0000 7f8012ffd700 1 -- 172.21.15.40:0/2840635113 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8014011d70 con 0x7f802405fa50 2020-11-20T13:56:50.971 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.957+0000 7f8012ffd700 1 -- 172.21.15.40:0/2840635113 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f801401f610 con 0x7f802405fa50 2020-11-20T13:56:50.972 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.957+0000 7f802bfb6700 1 -- 172.21.15.40:0/2840635113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80240af470 con 0x7f802405fa50 2020-11-20T13:56:50.972 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.957+0000 7f802bfb6700 1 -- 172.21.15.40:0/2840635113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8024147000 con 0x7f802405fa50 2020-11-20T13:56:50.972 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.958+0000 7f8012ffd700 1 -- 172.21.15.40:0/2840635113 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f8014026030 con 0x7f802405fa50 2020-11-20T13:56:50.972 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.958+0000 7f802bfb6700 1 -- 172.21.15.40:0/2840635113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f802413eb70 con 0x7f802405fa50 2020-11-20T13:56:50.972 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.958+0000 7f8012ffd700 1 --2- 172.21.15.40:0/2840635113 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f8004051290 0x7f8004053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:50.972 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.958+0000 7f8012ffd700 1 -- 172.21.15.40:0/2840635113 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f8014069840 con 0x7f802405fa50 2020-11-20T13:56:50.973 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.958+0000 7f8029551700 1 --2- 172.21.15.40:0/2840635113 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f8004051290 0x7f8004053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:50.973 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.958+0000 7f8029551700 1 --2- 172.21.15.40:0/2840635113 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f8004051290 0x7f8004053750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f802000d000 tx=0x7f8020007930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:50.973 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:50.962+0000 7f8012ffd700 1 -- 172.21.15.40:0/2840635113 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f8014041a60 con 0x7f802405fa50 2020-11-20T13:56:51.122 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.121+0000 7f802bfb6700 1 -- 172.21.15.40:0/2840635113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f8024057050 con 0x7f802405fa50 2020-11-20T13:56:51.122 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.121+0000 7f8012ffd700 1 -- 172.21.15.40:0/2840635113 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v24) v1 ==== 110+0+10 (secure 0 0 0) 0x7f80140240b0 con 0x7f802405fa50 2020-11-20T13:56:51.123 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2020-11-20T13:56:51.124 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.122+0000 7f802bfb6700 1 -- 172.21.15.40:0/2840635113 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f8004051290 msgr2=0x7f8004053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:51.124 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.122+0000 7f802bfb6700 1 --2- 172.21.15.40:0/2840635113 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f8004051290 0x7f8004053750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f802000d000 tx=0x7f8020007930).stop 2020-11-20T13:56:51.124 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.122+0000 7f802bfb6700 1 -- 172.21.15.40:0/2840635113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f802405fa50 msgr2=0x7f8024067820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:51.124 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.122+0000 7f802bfb6700 1 --2- 172.21.15.40:0/2840635113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f802405fa50 0x7f8024067820 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f8014002f30 tx=0x7f8014016000).stop 2020-11-20T13:56:51.125 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.122+0000 7f802bfb6700 1 -- 172.21.15.40:0/2840635113 shutdown_connections 2020-11-20T13:56:51.125 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.122+0000 7f802bfb6700 1 --2- 172.21.15.40:0/2840635113 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f8004051290 0x7f8004053750 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:51.126 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.122+0000 7f802bfb6700 1 --2- 172.21.15.40:0/2840635113 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f802413f400 0x7f802413e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:51.126 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.122+0000 7f802bfb6700 1 --2- 172.21.15.40:0/2840635113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f802405fa50 0x7f8024067820 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:51.126 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.122+0000 7f802bfb6700 1 --2- 172.21.15.40:0/2840635113 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f80240b6900 0x7f80240636e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:51.126 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.122+0000 7f802bfb6700 1 -- 172.21.15.40:0/2840635113 >> 172.21.15.40:0/2840635113 conn(0x7f80240ba1d0 msgr2=0x7f80240b7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:51.127 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.122+0000 7f802bfb6700 1 -- 172.21.15.40:0/2840635113 shutdown_connections 2020-11-20T13:56:51.127 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.123+0000 7f802bfb6700 1 -- 172.21.15.40:0/2840635113 wait complete. 2020-11-20T13:56:51.134 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-20T13:56:51.251 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.250+0000 7ff90e5b8700 1 Processor -- start 2020-11-20T13:56:51.251 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.250+0000 7ff90e5b8700 1 -- start start 2020-11-20T13:56:51.253 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.251+0000 7ff90e5b8700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff9080b31f0 0x7ff908144bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:51.254 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.251+0000 7ff90e5b8700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff9080b1450 0x7ff9080b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:51.254 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.251+0000 7ff90e5b8700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9080b0d10 con 0x7ff9080b6c80 2020-11-20T13:56:51.254 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.251+0000 7ff90e5b8700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9080b0fb0 con 0x7ff9080b31f0 2020-11-20T13:56:51.255 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.251+0000 7ff90e5b8700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9080b1da0 con 0x7ff9080b1450 2020-11-20T13:56:51.255 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.251+0000 7ff907fff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff9080b6c80 0x7ff9080b40f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46064/0 (socket says 172.21.15.40:46064) 2020-11-20T13:56:51.255 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.251+0000 7ff907fff700 1 -- 172.21.15.40:0/3853095931 learned_addr learned my addr 172.21.15.40:0/3853095931 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:51.255 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.251+0000 7ff9077fe700 1 --2- 172.21.15.40:0/3853095931 >> v2:172.21.15.116:3300/0 conn(0x7ff9080b31f0 0x7ff908144bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:51.256 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.251+0000 7ff90cb55700 1 --2- 172.21.15.40:0/3853095931 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff9080b1450 0x7ff9080b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:51.256 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.251+0000 7ff906ffd700 1 -- 172.21.15.40:0/3853095931 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3813769469 0 0) 0x7ff9080b0d10 con 0x7ff9080b6c80 2020-11-20T13:56:51.257 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.251+0000 7ff906ffd700 1 -- 172.21.15.40:0/3853095931 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8f4003230 con 0x7ff9080b6c80 2020-11-20T13:56:51.257 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.251+0000 7ff90cb55700 1 -- 172.21.15.40:0/3853095931 >> v2:172.21.15.116:3300/0 conn(0x7ff9080b31f0 msgr2=0x7ff908144bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:51.257 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.251+0000 7ff90cb55700 1 --2- 172.21.15.40:0/3853095931 >> v2:172.21.15.116:3300/0 conn(0x7ff9080b31f0 0x7ff908144bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:51.257 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.251+0000 7ff90cb55700 1 -- 172.21.15.40:0/3853095931 >> v1:172.21.15.40:6789/0 conn(0x7ff9080b6c80 legacy=0x7ff9080b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:51.258 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.251+0000 7ff90cb55700 1 -- 172.21.15.40:0/3853095931 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9080b7060 con 0x7ff9080b1450 2020-11-20T13:56:51.258 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.251+0000 7ff9077fe700 1 --2- 172.21.15.40:0/3853095931 >> v2:172.21.15.116:3300/0 conn(0x7ff9080b31f0 0x7ff908144bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:56:51.258 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.251+0000 7ff90cb55700 1 --2- 172.21.15.40:0/3853095931 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff9080b1450 0x7ff9080b18b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7ff9000032b0 tx=0x7ff9000091a0).ready entity=mon.2 client_cookie=3fd3101b74434ad server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:51.258 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.252+0000 7ff906ffd700 1 -- 172.21.15.40:0/3853095931 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff90000b940 con 0x7ff9080b1450 2020-11-20T13:56:51.259 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.252+0000 7ff906ffd700 1 -- 172.21.15.40:0/3853095931 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff90000baa0 con 0x7ff9080b1450 2020-11-20T13:56:51.260 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.252+0000 7ff906ffd700 1 -- 172.21.15.40:0/3853095931 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff90000bc10 con 0x7ff9080b1450 2020-11-20T13:56:51.260 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.252+0000 7ff90e5b8700 1 -- 172.21.15.40:0/3853095931 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff9080b1450 msgr2=0x7ff9080b18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:51.260 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.252+0000 7ff90e5b8700 1 --2- 172.21.15.40:0/3853095931 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff9080b1450 0x7ff9080b18b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7ff9000032b0 tx=0x7ff9000091a0).stop 2020-11-20T13:56:51.260 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.252+0000 7ff90e5b8700 1 -- 172.21.15.40:0/3853095931 shutdown_connections 2020-11-20T13:56:51.260 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.252+0000 7ff90e5b8700 1 --2- 172.21.15.40:0/3853095931 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff9080b1450 0x7ff9080b18b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:51.261 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.252+0000 7ff90e5b8700 1 --2- 172.21.15.40:0/3853095931 >> v2:172.21.15.116:3300/0 conn(0x7ff9080b31f0 0x7ff908144bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:51.261 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.252+0000 7ff90e5b8700 1 -- 172.21.15.40:0/3853095931 >> 172.21.15.40:0/3853095931 conn(0x7ff9080ba1d0 msgr2=0x7ff9080b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:51.261 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.252+0000 7ff90e5b8700 1 -- 172.21.15.40:0/3853095931 shutdown_connections 2020-11-20T13:56:51.262 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.252+0000 7ff90e5b8700 1 -- 172.21.15.40:0/3853095931 wait complete. 2020-11-20T13:56:51.264 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.253+0000 7ff90e5b8700 1 Processor -- start 2020-11-20T13:56:51.264 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.253+0000 7ff90e5b8700 1 -- start start 2020-11-20T13:56:51.265 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.254+0000 7ff90e5b8700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff9080b1450 0x7ff908067060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:51.265 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.254+0000 7ff90e5b8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9080b31f0 0x7ff908156820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:51.265 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.254+0000 7ff90e5b8700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9080b6c80 0x7ff908156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:51.265 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.254+0000 7ff90e5b8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff908066960 con 0x7ff9080b31f0 2020-11-20T13:56:51.266 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.254+0000 7ff90e5b8700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff908066bc0 con 0x7ff9080b6c80 2020-11-20T13:56:51.266 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.254+0000 7ff90e5b8700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9081573a0 con 0x7ff9080b1450 2020-11-20T13:56:51.267 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.254+0000 7ff9077fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9080b31f0 0x7ff908156820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:51.267 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.254+0000 7ff9077fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9080b31f0 0x7ff908156820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55474/0 (socket says 172.21.15.40:55474) 2020-11-20T13:56:51.267 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.254+0000 7ff9077fe700 1 -- 172.21.15.40:0/219879277 learned_addr learned my addr 172.21.15.40:0/219879277 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:51.268 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.254+0000 7ff907fff700 1 --2- 172.21.15.40:0/219879277 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff9080b1450 0x7ff908067060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:51.268 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.254+0000 7ff90cb55700 1 --2- 172.21.15.40:0/219879277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9080b6c80 0x7ff908156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:51.268 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.254+0000 7ff9077fe700 1 -- 172.21.15.40:0/219879277 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff9080b1450 msgr2=0x7ff908067060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:51.268 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.254+0000 7ff9077fe700 1 --2- 172.21.15.40:0/219879277 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff9080b1450 0x7ff908067060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:51.269 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.254+0000 7ff9077fe700 1 -- 172.21.15.40:0/219879277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9080b6c80 msgr2=0x7ff908156d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:51.269 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.254+0000 7ff9077fe700 1 --2- 172.21.15.40:0/219879277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9080b6c80 0x7ff908156d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:51.269 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.254+0000 7ff9077fe700 1 -- 172.21.15.40:0/219879277 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff908064a70 con 0x7ff9080b31f0 2020-11-20T13:56:51.269 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.255+0000 7ff9077fe700 1 --2- 172.21.15.40:0/219879277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9080b31f0 0x7ff908156820 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7ff8fc00c000 tx=0x7ff8fc006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:51.269 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.255+0000 7ff9057fa700 1 -- 172.21.15.40:0/219879277 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8fc00cae0 con 0x7ff9080b31f0 2020-11-20T13:56:51.270 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.255+0000 7ff9057fa700 1 -- 172.21.15.40:0/219879277 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8fc00cc40 con 0x7ff9080b31f0 2020-11-20T13:56:51.270 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.255+0000 7ff9057fa700 1 -- 172.21.15.40:0/219879277 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8fc0123f0 con 0x7ff9080b31f0 2020-11-20T13:56:51.270 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.255+0000 7ff90e5b8700 1 -- 172.21.15.40:0/219879277 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff908067550 con 0x7ff9080b31f0 2020-11-20T13:56:51.271 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.255+0000 7ff90e5b8700 1 -- 172.21.15.40:0/219879277 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9080b0250 con 0x7ff9080b31f0 2020-11-20T13:56:51.271 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.256+0000 7ff90e5b8700 1 -- 172.21.15.40:0/219879277 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff908157200 con 0x7ff9080b31f0 2020-11-20T13:56:51.271 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.256+0000 7ff9057fa700 1 -- 172.21.15.40:0/219879277 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7ff8fc012550 con 0x7ff9080b31f0 2020-11-20T13:56:51.271 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.256+0000 7ff9057fa700 1 --2- 172.21.15.40:0/219879277 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7ff8e8051320 0x7ff8e80537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:51.272 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.256+0000 7ff9057fa700 1 -- 172.21.15.40:0/219879277 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7ff8fc069770 con 0x7ff9080b31f0 2020-11-20T13:56:51.272 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.256+0000 7ff907fff700 1 --2- 172.21.15.40:0/219879277 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7ff8e8051320 0x7ff8e80537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:51.272 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.257+0000 7ff907fff700 1 --2- 172.21.15.40:0/219879277 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7ff8e8051320 0x7ff8e80537e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7ff8f800d000 tx=0x7ff8f8002720).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:51.272 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.260+0000 7ff9057fa700 1 -- 172.21.15.40:0/219879277 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7ff908157200 con 0x7ff9080b31f0 2020-11-20T13:56:51.422 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.421+0000 7ff90e5b8700 1 -- 172.21.15.40:0/219879277 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff908157200 con 0x7ff9080b31f0 2020-11-20T13:56:51.422 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.421+0000 7ff9057fa700 1 -- 172.21.15.40:0/219879277 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff908157200 con 0x7ff9080b31f0 2020-11-20T13:56:51.423 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2020-11-20T13:56:51.425 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.423+0000 7ff90e5b8700 1 -- 172.21.15.40:0/219879277 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7ff8e8051320 msgr2=0x7ff8e80537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:51.425 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.423+0000 7ff90e5b8700 1 --2- 172.21.15.40:0/219879277 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7ff8e8051320 0x7ff8e80537e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7ff8f800d000 tx=0x7ff8f8002720).stop 2020-11-20T13:56:51.425 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.423+0000 7ff90e5b8700 1 -- 172.21.15.40:0/219879277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9080b31f0 msgr2=0x7ff908156820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:51.425 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.423+0000 7ff90e5b8700 1 --2- 172.21.15.40:0/219879277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9080b31f0 0x7ff908156820 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7ff8fc00c000 tx=0x7ff8fc006dc0).stop 2020-11-20T13:56:51.426 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.423+0000 7ff90e5b8700 1 -- 172.21.15.40:0/219879277 shutdown_connections 2020-11-20T13:56:51.426 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.423+0000 7ff90e5b8700 1 --2- 172.21.15.40:0/219879277 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7ff8e8051320 0x7ff8e80537e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:51.426 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.423+0000 7ff90e5b8700 1 --2- 172.21.15.40:0/219879277 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff9080b1450 0x7ff908067060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:51.426 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.423+0000 7ff90e5b8700 1 --2- 172.21.15.40:0/219879277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9080b6c80 0x7ff908156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:51.426 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.423+0000 7ff90e5b8700 1 --2- 172.21.15.40:0/219879277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9080b31f0 0x7ff908156820 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:51.427 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.423+0000 7ff90e5b8700 1 -- 172.21.15.40:0/219879277 >> 172.21.15.40:0/219879277 conn(0x7ff9080ba1d0 msgr2=0x7ff9080b84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:51.427 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.423+0000 7ff90e5b8700 1 -- 172.21.15.40:0/219879277 shutdown_connections 2020-11-20T13:56:51.427 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.423+0000 7ff90e5b8700 1 -- 172.21.15.40:0/219879277 wait complete. 2020-11-20T13:56:51.434 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-20T13:56:51.551 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.549+0000 7f3dc909a700 1 Processor -- start 2020-11-20T13:56:51.551 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.549+0000 7f3dc909a700 1 -- start start 2020-11-20T13:56:51.554 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.550+0000 7f3dc909a700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3dc4060510 0x7f3dc4062030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:51.554 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.550+0000 7f3dc909a700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3dc40656a0 0x7f3dc41473e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:51.554 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.550+0000 7f3dc909a700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dc4064cc0 con 0x7f3dc405f710 2020-11-20T13:56:51.554 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.550+0000 7f3dc909a700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dc4064f60 con 0x7f3dc40656a0 2020-11-20T13:56:51.555 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.550+0000 7f3dc909a700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc4065200 con 0x7f3dc4060510 2020-11-20T13:56:51.555 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.550+0000 7f3dc2d9d700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3dc405f710 0x7f3dc412cfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46076/0 (socket says 172.21.15.40:46076) 2020-11-20T13:56:51.555 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.550+0000 7f3dc2d9d700 1 -- 172.21.15.40:0/202015774 learned_addr learned my addr 172.21.15.40:0/202015774 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:51.555 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.550+0000 7f3dc259c700 1 --2- 172.21.15.40:0/202015774 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3dc4060510 0x7f3dc4062030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:51.555 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.551+0000 7f3dc359e700 1 --2- 172.21.15.40:0/202015774 >> v2:172.21.15.116:3300/0 conn(0x7f3dc40656a0 0x7f3dc41473e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:51.556 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.551+0000 7f3dc1d9b700 1 -- 172.21.15.40:0/202015774 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2788979967 0 0) 0x7f3dc4064cc0 con 0x7f3dc405f710 2020-11-20T13:56:51.556 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.551+0000 7f3dc1d9b700 1 -- 172.21.15.40:0/202015774 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3da8003230 con 0x7f3dc405f710 2020-11-20T13:56:51.556 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.551+0000 7f3dc259c700 1 -- 172.21.15.40:0/202015774 >> v2:172.21.15.116:3300/0 conn(0x7f3dc40656a0 msgr2=0x7f3dc41473e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:51.556 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.551+0000 7f3dc259c700 1 --2- 172.21.15.40:0/202015774 >> v2:172.21.15.116:3300/0 conn(0x7f3dc40656a0 0x7f3dc41473e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:51.556 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.551+0000 7f3dc259c700 1 -- 172.21.15.40:0/202015774 >> v1:172.21.15.40:6789/0 conn(0x7f3dc405f710 legacy=0x7f3dc412cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:51.557 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.551+0000 7f3dc1d9b700 1 -- 172.21.15.40:0/202015774 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 987547732 0 0) 0x7f3da8003230 con 0x7f3dc405f710 2020-11-20T13:56:51.557 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.551+0000 7f3dc259c700 1 -- 172.21.15.40:0/202015774 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc405faf0 con 0x7f3dc4060510 2020-11-20T13:56:51.557 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.551+0000 7f3dc359e700 1 --2- 172.21.15.40:0/202015774 >> v2:172.21.15.116:3300/0 conn(0x7f3dc40656a0 0x7f3dc41473e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:56:51.557 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.551+0000 7f3dc259c700 1 --2- 172.21.15.40:0/202015774 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3dc4060510 0x7f3dc4062030 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f3db8001870 tx=0x7f3db4006dc0).ready entity=mon.2 client_cookie=c1e1e68f244b29b3 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:51.557 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.551+0000 7f3dc1d9b700 1 -- 172.21.15.40:0/202015774 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3db80016d0 con 0x7f3dc4060510 2020-11-20T13:56:51.558 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.551+0000 7f3dc1d9b700 1 -- 172.21.15.40:0/202015774 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db400d570 con 0x7f3dc4060510 2020-11-20T13:56:51.558 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.551+0000 7f3dc1d9b700 1 -- 172.21.15.40:0/202015774 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3db400d6e0 con 0x7f3dc4060510 2020-11-20T13:56:51.558 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.551+0000 7f3dc909a700 1 -- 172.21.15.40:0/202015774 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3dc4060510 msgr2=0x7f3dc4062030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:51.558 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.551+0000 7f3dc909a700 1 --2- 172.21.15.40:0/202015774 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3dc4060510 0x7f3dc4062030 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f3db8001870 tx=0x7f3db4006dc0).stop 2020-11-20T13:56:51.558 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.552+0000 7f3dc909a700 1 -- 172.21.15.40:0/202015774 shutdown_connections 2020-11-20T13:56:51.559 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.552+0000 7f3dc909a700 1 --2- 172.21.15.40:0/202015774 >> v2:172.21.15.116:3300/0 conn(0x7f3dc40656a0 0x7f3dc41473e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:51.559 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.552+0000 7f3dc909a700 1 --2- 172.21.15.40:0/202015774 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3dc4060510 0x7f3dc4062030 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:51.559 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.552+0000 7f3dc909a700 1 -- 172.21.15.40:0/202015774 >> 172.21.15.40:0/202015774 conn(0x7f3dc405cc60 msgr2=0x7f3dc40bb930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:51.559 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.552+0000 7f3dc909a700 1 -- 172.21.15.40:0/202015774 shutdown_connections 2020-11-20T13:56:51.559 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.552+0000 7f3dc909a700 1 -- 172.21.15.40:0/202015774 wait complete. 2020-11-20T13:56:51.560 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.553+0000 7f3dc909a700 1 Processor -- start 2020-11-20T13:56:51.560 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.553+0000 7f3dc909a700 1 -- start start 2020-11-20T13:56:51.564 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.553+0000 7f3dc909a700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3dc405f710 0x7f3dc415a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:51.564 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.553+0000 7f3dc909a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3dc4060510 0x7f3dc40b4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:51.564 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.553+0000 7f3dc909a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3dc40656a0 0x7f3dc40b1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:51.565 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.553+0000 7f3dc909a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc40b1da0 con 0x7f3dc4060510 2020-11-20T13:56:51.565 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.553+0000 7f3dc909a700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc40b2040 con 0x7f3dc40656a0 2020-11-20T13:56:51.565 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.553+0000 7f3dc909a700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc40b22e0 con 0x7f3dc405f710 2020-11-20T13:56:51.565 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.553+0000 7f3dc259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3dc4060510 0x7f3dc40b4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:51.565 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.554+0000 7f3dc259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3dc4060510 0x7f3dc40b4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55488/0 (socket says 172.21.15.40:55488) 2020-11-20T13:56:51.566 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.554+0000 7f3dc259c700 1 -- 172.21.15.40:0/384141519 learned_addr learned my addr 172.21.15.40:0/384141519 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:51.566 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.554+0000 7f3dc359e700 1 --2- 172.21.15.40:0/384141519 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3dc40656a0 0x7f3dc40b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:51.566 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.554+0000 7f3dc2d9d700 1 --2- 172.21.15.40:0/384141519 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3dc405f710 0x7f3dc415a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:51.566 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.554+0000 7f3dc259c700 1 -- 172.21.15.40:0/384141519 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3dc405f710 msgr2=0x7f3dc415a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:51.566 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.554+0000 7f3dc259c700 1 --2- 172.21.15.40:0/384141519 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3dc405f710 0x7f3dc415a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:51.567 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.554+0000 7f3dc259c700 1 -- 172.21.15.40:0/384141519 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3dc40656a0 msgr2=0x7f3dc40b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:51.567 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.554+0000 7f3dc259c700 1 --2- 172.21.15.40:0/384141519 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3dc40656a0 0x7f3dc40b1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:51.567 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.554+0000 7f3dc259c700 1 -- 172.21.15.40:0/384141519 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc40b5900 con 0x7f3dc4060510 2020-11-20T13:56:51.567 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.554+0000 7f3dc359e700 1 --2- 172.21.15.40:0/384141519 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3dc40656a0 0x7f3dc40b1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:51.567 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.554+0000 7f3dc259c700 1 --2- 172.21.15.40:0/384141519 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3dc4060510 0x7f3dc40b4af0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f3db8001870 tx=0x7f3db4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:51.568 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.554+0000 7f3db3fff700 1 -- 172.21.15.40:0/384141519 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3db4011850 con 0x7f3dc4060510 2020-11-20T13:56:51.568 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.554+0000 7f3db3fff700 1 -- 172.21.15.40:0/384141519 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db40119b0 con 0x7f3dc4060510 2020-11-20T13:56:51.568 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.554+0000 7f3dc909a700 1 -- 172.21.15.40:0/384141519 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dc40b5a90 con 0x7f3dc4060510 2020-11-20T13:56:51.568 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.554+0000 7f3db3fff700 1 -- 172.21.15.40:0/384141519 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3db4011d10 con 0x7f3dc4060510 2020-11-20T13:56:51.568 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.554+0000 7f3dc909a700 1 -- 172.21.15.40:0/384141519 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dc4147530 con 0x7f3dc4060510 2020-11-20T13:56:51.569 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.555+0000 7f3dc909a700 1 -- 172.21.15.40:0/384141519 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dc40b1bf0 con 0x7f3dc4060510 2020-11-20T13:56:51.569 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.555+0000 7f3db3fff700 1 -- 172.21.15.40:0/384141519 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f3db4024030 con 0x7f3dc4060510 2020-11-20T13:56:51.569 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.555+0000 7f3db3fff700 1 --2- 172.21.15.40:0/384141519 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f3da00513c0 0x7f3da0053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:51.569 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.556+0000 7f3db3fff700 1 -- 172.21.15.40:0/384141519 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f3db4069e10 con 0x7f3dc4060510 2020-11-20T13:56:51.570 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.556+0000 7f3dc2d9d700 1 --2- 172.21.15.40:0/384141519 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f3da00513c0 0x7f3da0053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:51.570 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.556+0000 7f3dc2d9d700 1 --2- 172.21.15.40:0/384141519 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f3da00513c0 0x7f3da0053880 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f3db800b000 tx=0x7f3db8001d20).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:51.570 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.559+0000 7f3db3fff700 1 -- 172.21.15.40:0/384141519 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f3db4042030 con 0x7f3dc4060510 2020-11-20T13:56:51.722 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.721+0000 7f3dc909a700 1 -- 172.21.15.40:0/384141519 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3dc4057050 con 0x7f3dc4060510 2020-11-20T13:56:51.723 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.721+0000 7f3db3fff700 1 -- 172.21.15.40:0/384141519 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+10 (secure 0 0 0) 0x7f3db4022020 con 0x7f3dc4060510 2020-11-20T13:56:51.723 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2020-11-20T13:56:51.724 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.723+0000 7f3dc909a700 1 -- 172.21.15.40:0/384141519 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f3da00513c0 msgr2=0x7f3da0053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:51.724 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.723+0000 7f3dc909a700 1 --2- 172.21.15.40:0/384141519 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f3da00513c0 0x7f3da0053880 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f3db800b000 tx=0x7f3db8001d20).stop 2020-11-20T13:56:51.724 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.723+0000 7f3dc909a700 1 -- 172.21.15.40:0/384141519 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3dc4060510 msgr2=0x7f3dc40b4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:51.725 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.723+0000 7f3dc909a700 1 --2- 172.21.15.40:0/384141519 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3dc4060510 0x7f3dc40b4af0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f3db8001870 tx=0x7f3db4015000).stop 2020-11-20T13:56:51.725 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.723+0000 7f3dc909a700 1 -- 172.21.15.40:0/384141519 shutdown_connections 2020-11-20T13:56:51.725 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.723+0000 7f3dc909a700 1 --2- 172.21.15.40:0/384141519 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f3da00513c0 0x7f3da0053880 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:51.725 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.723+0000 7f3dc909a700 1 --2- 172.21.15.40:0/384141519 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3dc405f710 0x7f3dc415a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:51.726 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.723+0000 7f3dc909a700 1 --2- 172.21.15.40:0/384141519 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3dc40656a0 0x7f3dc40b1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:51.726 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.723+0000 7f3dc909a700 1 --2- 172.21.15.40:0/384141519 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3dc4060510 0x7f3dc40b4af0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:51.726 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.723+0000 7f3dc909a700 1 -- 172.21.15.40:0/384141519 >> 172.21.15.40:0/384141519 conn(0x7f3dc405cc60 msgr2=0x7f3dc40b7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:51.726 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.723+0000 7f3dc909a700 1 -- 172.21.15.40:0/384141519 shutdown_connections 2020-11-20T13:56:51.726 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.723+0000 7f3dc909a700 1 -- 172.21.15.40:0/384141519 wait complete. 2020-11-20T13:56:51.734 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-20T13:56:51.852 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.850+0000 7f65e9295700 1 Processor -- start 2020-11-20T13:56:51.852 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.850+0000 7f65e9295700 1 -- start start 2020-11-20T13:56:51.854 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.851+0000 7f65e9295700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f65e40b1b60 0x7f65e4144bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:51.854 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.851+0000 7f65e9295700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65e40b0d10 con 0x7f65e40b6c80 2020-11-20T13:56:51.854 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.851+0000 7f65e9295700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65e40b0fb0 con 0x7f65e40b31f0 2020-11-20T13:56:51.855 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.851+0000 7f65e9295700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65e40b7060 con 0x7f65e40b1b60 2020-11-20T13:56:51.855 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.851+0000 7f65e37fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f65e40b1b60 0x7f65e4144bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:51.855 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.852+0000 7f65e37fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f65e40b1b60 0x7f65e4144bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55498/0 (socket says 172.21.15.40:55498) 2020-11-20T13:56:51.855 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.852+0000 7f65e2ffd700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f65e40b6c80 0x7f65e40b40f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46090/0 (socket says 172.21.15.40:46090) 2020-11-20T13:56:51.855 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.852+0000 7f65e37fe700 1 -- 172.21.15.40:0/3406579729 learned_addr learned my addr 172.21.15.40:0/3406579729 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:51.856 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.852+0000 7f65e37fe700 1 -- 172.21.15.40:0/3406579729 >> v1:172.21.15.116:6789/0 conn(0x7f65e40b31f0 legacy=0x7f65e40b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:51.856 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.852+0000 7f65e37fe700 1 -- 172.21.15.40:0/3406579729 >> v1:172.21.15.40:6789/0 conn(0x7f65e40b6c80 legacy=0x7f65e40b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:51.856 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.852+0000 7f65e37fe700 1 -- 172.21.15.40:0/3406579729 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e40b4800 con 0x7f65e40b1b60 2020-11-20T13:56:51.856 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.852+0000 7f65e1ffb700 1 -- 172.21.15.40:0/3406579729 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2828950766 0 0) 0x7f65e40b0d10 con 0x7f65e40b6c80 2020-11-20T13:56:51.857 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.852+0000 7f65e37fe700 1 --2- 172.21.15.40:0/3406579729 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65e40b1b60 0x7f65e4144bd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f65dc0032b0 tx=0x7f65dc009270).ready entity=mon.0 client_cookie=ca397735dca07788 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:51.857 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.852+0000 7f65e1ffb700 1 -- 172.21.15.40:0/3406579729 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65dc00bc00 con 0x7f65e40b1b60 2020-11-20T13:56:51.857 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.852+0000 7f65e1ffb700 1 -- 172.21.15.40:0/3406579729 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65dc00bd60 con 0x7f65e40b1b60 2020-11-20T13:56:51.857 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.852+0000 7f65e1ffb700 1 -- 172.21.15.40:0/3406579729 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65dc011400 con 0x7f65e40b1b60 2020-11-20T13:56:51.857 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.852+0000 7f65e9295700 1 -- 172.21.15.40:0/3406579729 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65e40b1b60 msgr2=0x7f65e4144bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:51.858 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.852+0000 7f65e9295700 1 --2- 172.21.15.40:0/3406579729 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65e40b1b60 0x7f65e4144bd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f65dc0032b0 tx=0x7f65dc009270).stop 2020-11-20T13:56:51.858 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.853+0000 7f65e9295700 1 -- 172.21.15.40:0/3406579729 shutdown_connections 2020-11-20T13:56:51.858 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.853+0000 7f65e9295700 1 --2- 172.21.15.40:0/3406579729 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65e40b1b60 0x7f65e4144bd0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:51.858 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.853+0000 7f65e9295700 1 -- 172.21.15.40:0/3406579729 >> 172.21.15.40:0/3406579729 conn(0x7f65e40ba1d0 msgr2=0x7f65e40b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:51.858 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.853+0000 7f65e9295700 1 -- 172.21.15.40:0/3406579729 shutdown_connections 2020-11-20T13:56:51.859 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.853+0000 7f65e9295700 1 -- 172.21.15.40:0/3406579729 wait complete. 2020-11-20T13:56:51.862 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.854+0000 7f65e9295700 1 Processor -- start 2020-11-20T13:56:51.862 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.854+0000 7f65e9295700 1 -- start start 2020-11-20T13:56:51.862 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.854+0000 7f65e9295700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f65e40b1b60 0x7f65e4125a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:51.862 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.854+0000 7f65e9295700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f65e40b31f0 0x7f65e4143560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:51.863 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.854+0000 7f65e9295700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65e40b6c80 0x7f65e40aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:51.863 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.854+0000 7f65e9295700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e412c3b0 con 0x7f65e40b6c80 2020-11-20T13:56:51.863 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.854+0000 7f65e9295700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e4143a50 con 0x7f65e40b1b60 2020-11-20T13:56:51.863 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.854+0000 7f65e9295700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e4064650 con 0x7f65e40b31f0 2020-11-20T13:56:51.863 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.854+0000 7f65e37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65e40b6c80 0x7f65e40aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:51.864 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.855+0000 7f65e37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65e40b6c80 0x7f65e40aefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55504/0 (socket says 172.21.15.40:55504) 2020-11-20T13:56:51.864 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.855+0000 7f65e37fe700 1 -- 172.21.15.40:0/1485398745 learned_addr learned my addr 172.21.15.40:0/1485398745 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:51.864 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.855+0000 7f65e2ffd700 1 --2- 172.21.15.40:0/1485398745 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f65e40b1b60 0x7f65e4125a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:51.864 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.855+0000 7f65e27fc700 1 --2- 172.21.15.40:0/1485398745 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f65e40b31f0 0x7f65e4143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:51.864 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.855+0000 7f65e37fe700 1 -- 172.21.15.40:0/1485398745 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f65e40b31f0 msgr2=0x7f65e4143560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:51.865 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.855+0000 7f65e37fe700 1 --2- 172.21.15.40:0/1485398745 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f65e40b31f0 0x7f65e4143560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:51.865 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.855+0000 7f65e37fe700 1 -- 172.21.15.40:0/1485398745 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f65e40b1b60 msgr2=0x7f65e4125a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:51.865 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.855+0000 7f65e37fe700 1 --2- 172.21.15.40:0/1485398745 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f65e40b1b60 0x7f65e4125a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:51.865 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.855+0000 7f65e37fe700 1 -- 172.21.15.40:0/1485398745 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e412c110 con 0x7f65e40b6c80 2020-11-20T13:56:51.866 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.855+0000 7f65e27fc700 1 --2- 172.21.15.40:0/1485398745 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f65e40b31f0 0x7f65e4143560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:51.866 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.855+0000 7f65e2ffd700 1 --2- 172.21.15.40:0/1485398745 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f65e40b1b60 0x7f65e4125a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:51.866 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.855+0000 7f65e37fe700 1 --2- 172.21.15.40:0/1485398745 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65e40b6c80 0x7f65e40aefc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f65dc0032b0 tx=0x7f65dc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:51.866 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.855+0000 7f65dbfff700 1 -- 172.21.15.40:0/1485398745 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65dc0107a0 con 0x7f65e40b6c80 2020-11-20T13:56:51.866 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.855+0000 7f65dbfff700 1 -- 172.21.15.40:0/1485398745 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65dc010900 con 0x7f65e40b6c80 2020-11-20T13:56:51.867 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.855+0000 7f65dbfff700 1 -- 172.21.15.40:0/1485398745 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65dc010c60 con 0x7f65e40b6c80 2020-11-20T13:56:51.867 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.855+0000 7f65e9295700 1 -- 172.21.15.40:0/1485398745 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65e412c780 con 0x7f65e40b6c80 2020-11-20T13:56:51.867 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.855+0000 7f65e9295700 1 -- 172.21.15.40:0/1485398745 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65e40af720 con 0x7f65e40b6c80 2020-11-20T13:56:51.867 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.856+0000 7f65e9295700 1 -- 172.21.15.40:0/1485398745 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65e4057050 con 0x7f65e40b6c80 2020-11-20T13:56:51.867 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.856+0000 7f65dbfff700 1 -- 172.21.15.40:0/1485398745 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f65dc00f070 con 0x7f65e40b6c80 2020-11-20T13:56:51.868 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.856+0000 7f65dbfff700 1 --2- 172.21.15.40:0/1485398745 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f65c8051290 0x7f65c8053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:51.868 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.857+0000 7f65dbfff700 1 -- 172.21.15.40:0/1485398745 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f65dc06a3b0 con 0x7f65e40b6c80 2020-11-20T13:56:51.868 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.857+0000 7f65e2ffd700 1 --2- 172.21.15.40:0/1485398745 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f65c8051290 0x7f65c8053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:51.868 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.857+0000 7f65e2ffd700 1 --2- 172.21.15.40:0/1485398745 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f65c8051290 0x7f65c8053750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f65cc008000 tx=0x7f65cc009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:51.868 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:51.860+0000 7f65dbfff700 1 -- 172.21.15.40:0/1485398745 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f65dc0425d0 con 0x7f65e40b6c80 2020-11-20T13:56:52.023 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.021+0000 7f65e9295700 1 -- 172.21.15.40:0/1485398745 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f65e405edb0 con 0x7f65e40b6c80 2020-11-20T13:56:52.023 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.022+0000 7f65dbfff700 1 -- 172.21.15.40:0/1485398745 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+10 (secure 0 0 0) 0x7f65dc0240a0 con 0x7f65e40b6c80 2020-11-20T13:56:52.024 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2020-11-20T13:56:52.025 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.023+0000 7f65e9295700 1 -- 172.21.15.40:0/1485398745 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f65c8051290 msgr2=0x7f65c8053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:52.025 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.023+0000 7f65e9295700 1 --2- 172.21.15.40:0/1485398745 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f65c8051290 0x7f65c8053750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f65cc008000 tx=0x7f65cc009040).stop 2020-11-20T13:56:52.025 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.023+0000 7f65e9295700 1 -- 172.21.15.40:0/1485398745 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65e40b6c80 msgr2=0x7f65e40aefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:52.026 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.023+0000 7f65e9295700 1 --2- 172.21.15.40:0/1485398745 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65e40b6c80 0x7f65e40aefc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f65dc0032b0 tx=0x7f65dc016000).stop 2020-11-20T13:56:52.026 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.023+0000 7f65e9295700 1 -- 172.21.15.40:0/1485398745 shutdown_connections 2020-11-20T13:56:52.026 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.023+0000 7f65e9295700 1 --2- 172.21.15.40:0/1485398745 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f65c8051290 0x7f65c8053750 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:52.026 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.023+0000 7f65e9295700 1 --2- 172.21.15.40:0/1485398745 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65e40b6c80 0x7f65e40aefc0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:52.026 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.023+0000 7f65e9295700 1 --2- 172.21.15.40:0/1485398745 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f65e40b1b60 0x7f65e4125a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:52.027 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.023+0000 7f65e9295700 1 --2- 172.21.15.40:0/1485398745 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f65e40b31f0 0x7f65e4143560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:52.027 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.023+0000 7f65e9295700 1 -- 172.21.15.40:0/1485398745 >> 172.21.15.40:0/1485398745 conn(0x7f65e40ba1d0 msgr2=0x7f65e40b8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:52.027 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.023+0000 7f65e9295700 1 -- 172.21.15.40:0/1485398745 shutdown_connections 2020-11-20T13:56:52.027 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.023+0000 7f65e9295700 1 -- 172.21.15.40:0/1485398745 wait complete. 2020-11-20T13:56:52.035 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-20T13:56:52.149 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.148+0000 7f5e33ef4700 1 Processor -- start 2020-11-20T13:56:52.150 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.148+0000 7f5e33ef4700 1 -- start start 2020-11-20T13:56:52.152 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.149+0000 7f5e33ef4700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5e2c0b88c0 0x7f5e2c0b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:52.152 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.149+0000 7f5e33ef4700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5e2c09a280 0x7f5e2c09c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:52.153 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.149+0000 7f5e33ef4700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e2c0b6be0 con 0x7f5e2c09b530 2020-11-20T13:56:52.153 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.149+0000 7f5e33ef4700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e2c099000 con 0x7f5e2c09a280 2020-11-20T13:56:52.153 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.149+0000 7f5e33ef4700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e2c0b4250 con 0x7f5e2c0b88c0 2020-11-20T13:56:52.153 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.149+0000 7f5e32491700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5e2c09a280 0x7f5e2c09c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:52.154 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.149+0000 7f5e32491700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5e2c09a280 0x7f5e2c09c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55512/0 (socket says 172.21.15.40:55512) 2020-11-20T13:56:52.154 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.149+0000 7f5e32491700 1 -- 172.21.15.40:0/1295239807 learned_addr learned my addr 172.21.15.40:0/1295239807 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:52.154 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.149+0000 7f5e31c90700 1 --2- 172.21.15.40:0/1295239807 >> v2:172.21.15.116:3300/0 conn(0x7f5e2c0b88c0 0x7f5e2c0b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:52.154 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.149+0000 7f5e32491700 1 -- 172.21.15.40:0/1295239807 >> v2:172.21.15.116:3300/0 conn(0x7f5e2c0b88c0 msgr2=0x7f5e2c0b8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:52.155 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.149+0000 7f5e32491700 1 --2- 172.21.15.40:0/1295239807 >> v2:172.21.15.116:3300/0 conn(0x7f5e2c0b88c0 0x7f5e2c0b8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:52.155 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.149+0000 7f5e32491700 1 -- 172.21.15.40:0/1295239807 >> v1:172.21.15.116:6789/0 conn(0x7f5e2c09b530 legacy=0x7f5e2c09b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:52.155 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.149+0000 7f5e32491700 1 -- 172.21.15.40:0/1295239807 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e2c0b43c0 con 0x7f5e2c09a280 2020-11-20T13:56:52.156 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.149+0000 7f5e31c90700 1 --2- 172.21.15.40:0/1295239807 >> v2:172.21.15.116:3300/0 conn(0x7f5e2c0b88c0 0x7f5e2c0b8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:52.156 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.149+0000 7f5e32491700 1 --2- 172.21.15.40:0/1295239807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e2c09a280 0x7f5e2c09c0d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f5e240032b0 tx=0x7f5e24009270).ready entity=mon.0 client_cookie=d0b652bda10e49da server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:52.157 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.149+0000 7f5e30c8e700 1 -- 172.21.15.40:0/1295239807 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e2400bc00 con 0x7f5e2c09a280 2020-11-20T13:56:52.157 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.149+0000 7f5e30c8e700 1 -- 172.21.15.40:0/1295239807 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e2400bd60 con 0x7f5e2c09a280 2020-11-20T13:56:52.157 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.149+0000 7f5e30c8e700 1 -- 172.21.15.40:0/1295239807 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e24011400 con 0x7f5e2c09a280 2020-11-20T13:56:52.157 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.150+0000 7f5e33ef4700 1 -- 172.21.15.40:0/1295239807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e2c09a280 msgr2=0x7f5e2c09c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:52.158 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.150+0000 7f5e33ef4700 1 --2- 172.21.15.40:0/1295239807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e2c09a280 0x7f5e2c09c0d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f5e240032b0 tx=0x7f5e24009270).stop 2020-11-20T13:56:52.158 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.150+0000 7f5e33ef4700 1 -- 172.21.15.40:0/1295239807 shutdown_connections 2020-11-20T13:56:52.158 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.150+0000 7f5e33ef4700 1 --2- 172.21.15.40:0/1295239807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e2c09a280 0x7f5e2c09c0d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:52.158 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.150+0000 7f5e33ef4700 1 --2- 172.21.15.40:0/1295239807 >> v2:172.21.15.116:3300/0 conn(0x7f5e2c0b88c0 0x7f5e2c0b8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:52.158 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.150+0000 7f5e33ef4700 1 -- 172.21.15.40:0/1295239807 >> 172.21.15.40:0/1295239807 conn(0x7f5e2c0ba380 msgr2=0x7f5e2c0b6350 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:52.159 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.150+0000 7f5e33ef4700 1 -- 172.21.15.40:0/1295239807 shutdown_connections 2020-11-20T13:56:52.159 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.150+0000 7f5e33ef4700 1 -- 172.21.15.40:0/1295239807 wait complete. 2020-11-20T13:56:52.160 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.151+0000 7f5e33ef4700 1 Processor -- start 2020-11-20T13:56:52.160 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.151+0000 7f5e33ef4700 1 -- start start 2020-11-20T13:56:52.162 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.152+0000 7f5e33ef4700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5e2c09a280 0x7f5e2c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:52.162 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.152+0000 7f5e33ef4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e2c09b530 0x7f5e2c148640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:52.163 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.152+0000 7f5e33ef4700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5e2c0b88c0 0x7f5e2c146e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:52.163 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.152+0000 7f5e33ef4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c148b30 con 0x7f5e2c09b530 2020-11-20T13:56:52.163 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.152+0000 7f5e33ef4700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c148ff0 con 0x7f5e2c0b88c0 2020-11-20T13:56:52.163 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.152+0000 7f5e33ef4700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c148d50 con 0x7f5e2c09a280 2020-11-20T13:56:52.164 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.152+0000 7f5e3148f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e2c09b530 0x7f5e2c148640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:52.164 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.152+0000 7f5e3148f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e2c09b530 0x7f5e2c148640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55516/0 (socket says 172.21.15.40:55516) 2020-11-20T13:56:52.164 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.152+0000 7f5e3148f700 1 -- 172.21.15.40:0/904631123 learned_addr learned my addr 172.21.15.40:0/904631123 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:52.164 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.152+0000 7f5e31c90700 1 --2- 172.21.15.40:0/904631123 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5e2c09a280 0x7f5e2c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:52.164 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.152+0000 7f5e32491700 1 --2- 172.21.15.40:0/904631123 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5e2c0b88c0 0x7f5e2c146e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:52.165 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.152+0000 7f5e3148f700 1 -- 172.21.15.40:0/904631123 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5e2c09a280 msgr2=0x7f5e2c062d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:52.165 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.152+0000 7f5e3148f700 1 --2- 172.21.15.40:0/904631123 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5e2c09a280 0x7f5e2c062d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:52.165 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.152+0000 7f5e3148f700 1 -- 172.21.15.40:0/904631123 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5e2c0b88c0 msgr2=0x7f5e2c146e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:52.165 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.152+0000 7f5e3148f700 1 --2- 172.21.15.40:0/904631123 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5e2c0b88c0 0x7f5e2c146e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:52.166 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.152+0000 7f5e3148f700 1 -- 172.21.15.40:0/904631123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e2c1469e0 con 0x7f5e2c09b530 2020-11-20T13:56:52.166 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.152+0000 7f5e3148f700 1 --2- 172.21.15.40:0/904631123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e2c09b530 0x7f5e2c148640 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f5e2c0b73b0 tx=0x7f5e1c007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:52.166 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.152+0000 7f5e1affd700 1 -- 172.21.15.40:0/904631123 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e1c007240 con 0x7f5e2c09b530 2020-11-20T13:56:52.166 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.152+0000 7f5e1affd700 1 -- 172.21.15.40:0/904631123 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e1c0073a0 con 0x7f5e2c09b530 2020-11-20T13:56:52.166 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.152+0000 7f5e33ef4700 1 -- 172.21.15.40:0/904631123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e2c144320 con 0x7f5e2c09b530 2020-11-20T13:56:52.167 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.152+0000 7f5e1affd700 1 -- 172.21.15.40:0/904631123 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e1c0064e0 con 0x7f5e2c09b530 2020-11-20T13:56:52.168 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.152+0000 7f5e33ef4700 1 -- 172.21.15.40:0/904631123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e2c157670 con 0x7f5e2c09b530 2020-11-20T13:56:52.168 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.154+0000 7f5e33ef4700 1 -- 172.21.15.40:0/904631123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e2c146b70 con 0x7f5e2c09b530 2020-11-20T13:56:52.168 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.154+0000 7f5e1affd700 1 -- 172.21.15.40:0/904631123 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f5e1c010070 con 0x7f5e2c09b530 2020-11-20T13:56:52.169 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.154+0000 7f5e1affd700 1 --2- 172.21.15.40:0/904631123 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f5e0c051320 0x7f5e0c0537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:52.169 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.154+0000 7f5e1affd700 1 -- 172.21.15.40:0/904631123 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f5e1c0685b0 con 0x7f5e2c09b530 2020-11-20T13:56:52.169 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.154+0000 7f5e31c90700 1 --2- 172.21.15.40:0/904631123 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f5e0c051320 0x7f5e0c0537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:52.169 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.154+0000 7f5e31c90700 1 --2- 172.21.15.40:0/904631123 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f5e0c051320 0x7f5e0c0537e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f5e2800c000 tx=0x7f5e28008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:52.169 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.157+0000 7f5e1affd700 1 -- 172.21.15.40:0/904631123 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f5e1c0407d0 con 0x7f5e2c09b530 2020-11-20T13:56:52.318 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.317+0000 7f5e33ef4700 1 -- 172.21.15.40:0/904631123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e2c057050 con 0x7f5e2c09b530 2020-11-20T13:56:52.319 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.318+0000 7f5e1affd700 1 -- 172.21.15.40:0/904631123 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16347 (secure 0 0 0) 0x7f5e1c018020 con 0x7f5e2c09b530 2020-11-20T13:56:52.319 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:56:52.319 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":24,"fsid":"da1ed737-5174-430f-a3c9-56f6bbfad13c","created":"2020-11-20T13:56:04.990651+0000","modified":"2020-11-20T13:56:25.428032+0000","last_up_change":"2020-11-20T13:56:13.893026+0000","last_in_change":"2020-11-20T13:56:13.893026+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T13:56:08.330017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T13:56:17.939684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T13:56:20.563740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T13:56:23.062623+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9ea4c667-8938-46d6-a5ca-092d55c1f79e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":22825},{"type":"v1","addr":"172.21.15.40:6818","nonce":22825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":22825},{"type":"v1","addr":"172.21.15.40:6822","nonce":22825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":22825},{"type":"v1","addr":"172.21.15.40:6830","nonce":22825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":22825},{"type":"v1","addr":"172.21.15.40:6826","nonce":22825}]},"public_addr":"172.21.15.40:6818/22825","cluster_addr":"172.21.15.40:6822/22825","heartbeat_back_addr":"172.21.15.40:6830/22825","heartbeat_front_addr":"172.21.15.40:6826/22825","state":["exists","up"]},{"osd":1,"uuid":"515708d9-5087-4e55-8d14-cdea1c00eacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":22823},{"type":"v1","addr":"172.21.15.40:6801","nonce":22823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":22823},{"type":"v1","addr":"172.21.15.40:6803","nonce":22823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":22823},{"type":"v1","addr":"172.21.15.40:6807","nonce":22823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":22823},{"type":"v1","addr":"172.21.15.40:6805","nonce":22823}]},"public_addr":"172.21.15.40:6801/22823","cluster_addr":"172.21.15.40:6803/22823","heartbeat_back_addr":"172.21.15.40:6807/22823","heartbeat_front_addr":"172.21.15.40:6805/22823","state":["exists","up"]},{"osd":2,"uuid":"4d86bca7-743e-4983-b35f-36676c405b36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":22824},{"type":"v1","addr":"172.21.15.40:6809","nonce":22824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":22824},{"type":"v1","addr":"172.21.15.40:6811","nonce":22824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":22824},{"type":"v1","addr":"172.21.15.40:6815","nonce":22824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":22824},{"type":"v1","addr":"172.21.15.40:6813","nonce":22824}]},"public_addr":"172.21.15.40:6809/22824","cluster_addr":"172.21.15.40:6811/22824","heartbeat_back_addr":"172.21.15.40:6815/22824","heartbeat_front_addr":"172.21.15.40:6813/22824","state":["exists","up"]},{"osd":3,"uuid":"f4dc726b-6b34-4e8b-8eeb-87060b58002d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":22826},{"type":"v1","addr":"172.21.15.40:6819","nonce":22826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":22826},{"type":"v1","addr":"172.21.15.40:6823","nonce":22826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":22826},{"type":"v1","addr":"172.21.15.40:6831","nonce":22826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":22826},{"type":"v1","addr":"172.21.15.40:6827","nonce":22826}]},"public_addr":"172.21.15.40:6819/22826","cluster_addr":"172.21.15.40:6823/22826","heartbeat_back_addr":"172.21.15.40:6831/22826","heartbeat_front_addr":"172.21.15.40:6827/22826","state":["exists","up"]},{"osd":4,"uuid":"25a6d53b-d1f0-4498-b7d5-ffa38661d403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":23181},{"type":"v1","addr":"172.21.15.116:6819","nonce":23181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":23181},{"type":"v1","addr":"172.21.15.116:6821","nonce":23181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":23181},{"type":"v1","addr":"172.21.15.116:6825","nonce":23181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":23181},{"type":"v1","addr":"172.21.15.116:6823","nonce":23181}]},"public_addr":"172.21.15.116:6819/23181","cluster_addr":"172.21.15.116:6821/23181","heartbeat_back_addr":"172.21.15.116:6825/23181","heartbeat_front_addr":"172.21.15.116:6823/23181","state":["exists","up"]},{"osd":5,"uuid":"7d7a5af8-229b-4351-9a21-a3d509cd4635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":23171},{"type":"v1","addr":"172.21.15.116:6803","nonce":23171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":23171},{"type":"v1","addr":"172.21.15.116:6805","nonce":23171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":23171},{"type":"v1","addr":"172.21.15.116:6809","nonce":23171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":23171},{"type":"v1","addr":"172.21.15.116:6807","nonce":23171}]},"public_addr":"172.21.15.116:6803/23171","cluster_addr":"172.21.15.116:6805/23171","heartbeat_back_addr":"172.21.15.116:6809/23171","heartbeat_front_addr":"172.21.15.116:6807/23171","state":["exists","up"]},{"osd":6,"uuid":"8247d248-a140-4e36-9796-9847a97df58d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":23172},{"type":"v1","addr":"172.21.15.116:6811","nonce":23172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":23172},{"type":"v1","addr":"172.21.15.116:6813","nonce":23172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":23172},{"type":"v1","addr":"172.21.15.116:6817","nonce":23172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":23172},{"type":"v1","addr":"172.21.15.116:6815","nonce":23172}]},"public_addr":"172.21.15.116:6811/23172","cluster_addr":"172.21.15.116:6813/23172","heartbeat_back_addr":"172.21.15.116:6817/23172","heartbeat_front_addr":"172.21.15.116:6815/23172","state":["exists","up"]},{"osd":7,"uuid":"472803c4-70b8-499d-b18a-71aaeec6aab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":23196},{"type":"v1","addr":"172.21.15.116:6827","nonce":23196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":23196},{"type":"v1","addr":"172.21.15.116:6829","nonce":23196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":23196},{"type":"v1","addr":"172.21.15.116:6833","nonce":23196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":23196},{"type":"v1","addr":"172.21.15.116:6831","nonce":23196}]},"public_addr":"172.21.15.116:6827/23196","cluster_addr":"172.21.15.116:6829/23196","heartbeat_back_addr":"172.21.15.116:6833/23196","heartbeat_front_addr":"172.21.15.116:6831/23196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.029357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:11.989797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.091097+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.014766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.070860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.071864+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.063400+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.109028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-20T13:56:52.320 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.319+0000 7f5e33ef4700 1 -- 172.21.15.40:0/904631123 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f5e0c051320 msgr2=0x7f5e0c0537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:52.321 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.319+0000 7f5e33ef4700 1 --2- 172.21.15.40:0/904631123 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f5e0c051320 0x7f5e0c0537e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f5e2800c000 tx=0x7f5e28008040).stop 2020-11-20T13:56:52.321 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.319+0000 7f5e33ef4700 1 -- 172.21.15.40:0/904631123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e2c09b530 msgr2=0x7f5e2c148640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:52.321 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.319+0000 7f5e33ef4700 1 --2- 172.21.15.40:0/904631123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e2c09b530 0x7f5e2c148640 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f5e2c0b73b0 tx=0x7f5e1c007be0).stop 2020-11-20T13:56:52.321 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.319+0000 7f5e33ef4700 1 -- 172.21.15.40:0/904631123 shutdown_connections 2020-11-20T13:56:52.322 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.319+0000 7f5e33ef4700 1 --2- 172.21.15.40:0/904631123 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f5e0c051320 0x7f5e0c0537e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:52.322 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.319+0000 7f5e33ef4700 1 --2- 172.21.15.40:0/904631123 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5e2c09a280 0x7f5e2c062d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:52.322 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.319+0000 7f5e33ef4700 1 --2- 172.21.15.40:0/904631123 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5e2c0b88c0 0x7f5e2c146e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:52.322 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.319+0000 7f5e33ef4700 1 --2- 172.21.15.40:0/904631123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e2c09b530 0x7f5e2c148640 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:52.322 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.319+0000 7f5e33ef4700 1 -- 172.21.15.40:0/904631123 >> 172.21.15.40:0/904631123 conn(0x7f5e2c0ba380 msgr2=0x7f5e2c09a660 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:52.323 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.319+0000 7f5e33ef4700 1 -- 172.21.15.40:0/904631123 shutdown_connections 2020-11-20T13:56:52.323 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.319+0000 7f5e33ef4700 1 -- 172.21.15.40:0/904631123 wait complete. 2020-11-20T13:56:52.331 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T13:56:08.330017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T13:56:17.939684+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T13:56:20.563740+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T13:56:23.062623+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T13:56:52.332 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-20T13:56:52.449 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.448+0000 7f2d8886a700 1 Processor -- start 2020-11-20T13:56:52.449 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.448+0000 7f2d8886a700 1 -- start start 2020-11-20T13:56:52.451 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.448+0000 7f2d8886a700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2d840b6c80 0x7f2d840b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:52.452 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.448+0000 7f2d8886a700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2d84144bd0 0x7f2d840b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:52.452 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.448+0000 7f2d8886a700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d840b0d10 con 0x7f2d840b45e0 2020-11-20T13:56:52.452 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.448+0000 7f2d8886a700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d84145030 con 0x7f2d84144bd0 2020-11-20T13:56:52.452 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.448+0000 7f2d8886a700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d840b0fb0 con 0x7f2d840b6c80 2020-11-20T13:56:52.453 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.449+0000 7f2d81d9b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2d84144bd0 0x7f2d840b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:52.453 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.449+0000 7f2d82d9d700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2d840b45e0 0x7f2d840b1450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46118/0 (socket says 172.21.15.40:46118) 2020-11-20T13:56:52.453 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.449+0000 7f2d82d9d700 1 -- 172.21.15.40:0/123367197 learned_addr learned my addr 172.21.15.40:0/123367197 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:52.453 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.449+0000 7f2d8259c700 1 --2- 172.21.15.40:0/123367197 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2d840b6c80 0x7f2d840b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:52.454 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.449+0000 7f2d81d9b700 1 -- 172.21.15.40:0/123367197 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2d840b6c80 msgr2=0x7f2d840b31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:52.454 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.449+0000 7f2d81d9b700 1 --2- 172.21.15.40:0/123367197 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2d840b6c80 0x7f2d840b31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:52.454 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.449+0000 7f2d81d9b700 1 -- 172.21.15.40:0/123367197 >> v1:172.21.15.40:6789/0 conn(0x7f2d840b45e0 legacy=0x7f2d840b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:52.454 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.449+0000 7f2d81d9b700 1 -- 172.21.15.40:0/123367197 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d840b7060 con 0x7f2d84144bd0 2020-11-20T13:56:52.455 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.449+0000 7f2d8159a700 1 -- 172.21.15.40:0/123367197 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 12356348 0 0) 0x7f2d840b0d10 con 0x7f2d840b45e0 2020-11-20T13:56:52.455 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.449+0000 7f2d8259c700 1 --2- 172.21.15.40:0/123367197 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2d840b6c80 0x7f2d840b31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:52.455 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.449+0000 7f2d81d9b700 1 --2- 172.21.15.40:0/123367197 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d84144bd0 0x7f2d840b40f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f2d74002f30 tx=0x7f2d74006dc0).ready entity=mon.0 client_cookie=bcccbeded8de1850 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:52.455 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.449+0000 7f2d8159a700 1 -- 172.21.15.40:0/123367197 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d740097f0 con 0x7f2d84144bd0 2020-11-20T13:56:52.456 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.449+0000 7f2d8159a700 1 -- 172.21.15.40:0/123367197 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d74009950 con 0x7f2d84144bd0 2020-11-20T13:56:52.456 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.449+0000 7f2d8159a700 1 -- 172.21.15.40:0/123367197 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d74009cb0 con 0x7f2d84144bd0 2020-11-20T13:56:52.456 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.449+0000 7f2d8886a700 1 -- 172.21.15.40:0/123367197 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d84144bd0 msgr2=0x7f2d840b40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:52.456 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.449+0000 7f2d8886a700 1 --2- 172.21.15.40:0/123367197 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d84144bd0 0x7f2d840b40f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f2d74002f30 tx=0x7f2d74006dc0).stop 2020-11-20T13:56:52.456 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.450+0000 7f2d8886a700 1 -- 172.21.15.40:0/123367197 shutdown_connections 2020-11-20T13:56:52.457 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.450+0000 7f2d8886a700 1 --2- 172.21.15.40:0/123367197 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2d840b6c80 0x7f2d840b31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:52.457 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.450+0000 7f2d8886a700 1 --2- 172.21.15.40:0/123367197 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d84144bd0 0x7f2d840b40f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:52.458 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.450+0000 7f2d8886a700 1 -- 172.21.15.40:0/123367197 >> 172.21.15.40:0/123367197 conn(0x7f2d840ba1d0 msgr2=0x7f2d840b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:52.458 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.450+0000 7f2d8886a700 1 -- 172.21.15.40:0/123367197 shutdown_connections 2020-11-20T13:56:52.458 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.450+0000 7f2d8886a700 1 -- 172.21.15.40:0/123367197 wait complete. 2020-11-20T13:56:52.461 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.451+0000 7f2d8886a700 1 Processor -- start 2020-11-20T13:56:52.462 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.451+0000 7f2d8886a700 1 -- start start 2020-11-20T13:56:52.462 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.451+0000 7f2d8886a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d840b45e0 0x7f2d8405f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:52.462 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.451+0000 7f2d8886a700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2d840b6c80 0x7f2d84157060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:52.462 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.451+0000 7f2d8886a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d84144bd0 0x7f2d840aeec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:52.463 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.451+0000 7f2d8886a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d84157550 con 0x7f2d840b45e0 2020-11-20T13:56:52.463 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.451+0000 7f2d8886a700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d84156440 con 0x7f2d84144bd0 2020-11-20T13:56:52.463 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.451+0000 7f2d8886a700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d84159e30 con 0x7f2d840b6c80 2020-11-20T13:56:52.463 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.451+0000 7f2d8259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d840b45e0 0x7f2d8405f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:52.463 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.452+0000 7f2d8259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d840b45e0 0x7f2d8405f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55526/0 (socket says 172.21.15.40:55526) 2020-11-20T13:56:52.464 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.452+0000 7f2d8259c700 1 -- 172.21.15.40:0/1845668837 learned_addr learned my addr 172.21.15.40:0/1845668837 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:52.464 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.452+0000 7f2d81d9b700 1 --2- 172.21.15.40:0/1845668837 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2d840b6c80 0x7f2d84157060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:52.464 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.452+0000 7f2d8259c700 1 -- 172.21.15.40:0/1845668837 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2d840b6c80 msgr2=0x7f2d84157060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:52.464 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.452+0000 7f2d8259c700 1 --2- 172.21.15.40:0/1845668837 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2d840b6c80 0x7f2d84157060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:52.464 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.452+0000 7f2d8259c700 1 -- 172.21.15.40:0/1845668837 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d84144bd0 msgr2=0x7f2d840aeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:52.465 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.452+0000 7f2d8259c700 1 --2- 172.21.15.40:0/1845668837 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d84144bd0 0x7f2d840aeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T13:56:52.465 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.452+0000 7f2d8259c700 1 -- 172.21.15.40:0/1845668837 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d8415a0d0 con 0x7f2d840b45e0 2020-11-20T13:56:52.466 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.452+0000 7f2d8259c700 1 --2- 172.21.15.40:0/1845668837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d840b45e0 0x7f2d8405f710 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f2d84157690 tx=0x7f2d6c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:52.466 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.452+0000 7f2d7b7fe700 1 -- 172.21.15.40:0/1845668837 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d6c009920 con 0x7f2d840b45e0 2020-11-20T13:56:52.466 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.452+0000 7f2d7b7fe700 1 -- 172.21.15.40:0/1845668837 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6c009a80 con 0x7f2d840b45e0 2020-11-20T13:56:52.466 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.452+0000 7f2d8886a700 1 -- 172.21.15.40:0/1845668837 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d8415a3d0 con 0x7f2d840b45e0 2020-11-20T13:56:52.467 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.452+0000 7f2d7b7fe700 1 -- 172.21.15.40:0/1845668837 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d6c009de0 con 0x7f2d840b45e0 2020-11-20T13:56:52.467 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.452+0000 7f2d8886a700 1 -- 172.21.15.40:0/1845668837 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d8415a810 con 0x7f2d840b45e0 2020-11-20T13:56:52.468 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.453+0000 7f2d8886a700 1 -- 172.21.15.40:0/1845668837 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d84057050 con 0x7f2d840b45e0 2020-11-20T13:56:52.468 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.453+0000 7f2d7b7fe700 1 -- 172.21.15.40:0/1845668837 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f2d6c013480 con 0x7f2d840b45e0 2020-11-20T13:56:52.469 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.453+0000 7f2d7b7fe700 1 --2- 172.21.15.40:0/1845668837 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f2d680513c0 0x7f2d68053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:52.469 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.454+0000 7f2d7b7fe700 1 -- 172.21.15.40:0/1845668837 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f2d6c0697c0 con 0x7f2d840b45e0 2020-11-20T13:56:52.469 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.454+0000 7f2d81d9b700 1 --2- 172.21.15.40:0/1845668837 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f2d680513c0 0x7f2d68053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:52.469 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.454+0000 7f2d81d9b700 1 --2- 172.21.15.40:0/1845668837 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f2d680513c0 0x7f2d68053880 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f2d74019000 tx=0x7f2d74015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:52.469 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.457+0000 7f2d7b7fe700 1 -- 172.21.15.40:0/1845668837 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f2d6c041980 con 0x7f2d840b45e0 2020-11-20T13:56:52.617 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.615+0000 7f2d8886a700 1 -- 172.21.15.40:0/1845668837 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f2d841485e0 con 0x7f2d840b45e0 2020-11-20T13:56:52.617 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.616+0000 7f2d7b7fe700 1 -- 172.21.15.40:0/1845668837 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v24) v1 ==== 110+0+10 (secure 0 0 0) 0x7f2d6c044fa0 con 0x7f2d840b45e0 2020-11-20T13:56:52.617 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2020-11-20T13:56:52.618 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.617+0000 7f2d8886a700 1 -- 172.21.15.40:0/1845668837 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f2d680513c0 msgr2=0x7f2d68053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:52.619 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.617+0000 7f2d8886a700 1 --2- 172.21.15.40:0/1845668837 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f2d680513c0 0x7f2d68053880 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f2d74019000 tx=0x7f2d74015000).stop 2020-11-20T13:56:52.619 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.617+0000 7f2d8886a700 1 -- 172.21.15.40:0/1845668837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d840b45e0 msgr2=0x7f2d8405f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:52.619 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.617+0000 7f2d8886a700 1 --2- 172.21.15.40:0/1845668837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d840b45e0 0x7f2d8405f710 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f2d84157690 tx=0x7f2d6c008040).stop 2020-11-20T13:56:52.619 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.617+0000 7f2d8886a700 1 -- 172.21.15.40:0/1845668837 shutdown_connections 2020-11-20T13:56:52.619 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.617+0000 7f2d8886a700 1 --2- 172.21.15.40:0/1845668837 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f2d680513c0 0x7f2d68053880 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:52.620 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.617+0000 7f2d8886a700 1 --2- 172.21.15.40:0/1845668837 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d84144bd0 0x7f2d840aeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T13:56:52.620 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.617+0000 7f2d8886a700 1 --2- 172.21.15.40:0/1845668837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d840b45e0 0x7f2d8405f710 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:52.620 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.617+0000 7f2d8886a700 1 --2- 172.21.15.40:0/1845668837 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2d840b6c80 0x7f2d84157060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:52.620 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.617+0000 7f2d8886a700 1 -- 172.21.15.40:0/1845668837 >> 172.21.15.40:0/1845668837 conn(0x7f2d840ba1d0 msgr2=0x7f2d840b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:52.620 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.617+0000 7f2d8886a700 1 -- 172.21.15.40:0/1845668837 shutdown_connections 2020-11-20T13:56:52.621 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.617+0000 7f2d8886a700 1 -- 172.21.15.40:0/1845668837 wait complete. 2020-11-20T13:56:52.629 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-20T13:56:52.746 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.745+0000 7fe84d333700 1 Processor -- start 2020-11-20T13:56:52.747 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.745+0000 7fe84d333700 1 -- start start 2020-11-20T13:56:52.749 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.746+0000 7fe84d333700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe8480b6c80 0x7fe8480b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:52.749 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.746+0000 7fe84d333700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe8480b1450 0x7fe8480b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:52.749 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.746+0000 7fe84d333700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8480b0d10 con 0x7fe848144bd0 2020-11-20T13:56:52.750 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.746+0000 7fe84d333700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe848145060 con 0x7fe8480b6c80 2020-11-20T13:56:52.750 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.746+0000 7fe84d333700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8480b0fb0 con 0x7fe8480b1450 2020-11-20T13:56:52.750 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.746+0000 7fe846ffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe8480b6c80 0x7fe8480b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:52.750 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.746+0000 7fe846ffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe8480b6c80 0x7fe8480b31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55534/0 (socket says 172.21.15.40:55534) 2020-11-20T13:56:52.750 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.746+0000 7fe846ffd700 1 -- 172.21.15.40:0/4243316638 learned_addr learned my addr 172.21.15.40:0/4243316638 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:52.751 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.746+0000 7fe8477fe700 1 --2- 172.21.15.40:0/4243316638 >> v2:172.21.15.116:3300/0 conn(0x7fe8480b1450 0x7fe8480b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:52.751 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.747+0000 7fe846ffd700 1 -- 172.21.15.40:0/4243316638 >> v2:172.21.15.116:3300/0 conn(0x7fe8480b1450 msgr2=0x7fe8480b18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:52.751 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.747+0000 7fe846ffd700 1 --2- 172.21.15.40:0/4243316638 >> v2:172.21.15.116:3300/0 conn(0x7fe8480b1450 0x7fe8480b18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:52.752 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.747+0000 7fe846ffd700 1 -- 172.21.15.40:0/4243316638 >> v1:172.21.15.116:6789/0 conn(0x7fe848144bd0 legacy=0x7fe8480b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:52.752 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.747+0000 7fe846ffd700 1 -- 172.21.15.40:0/4243316638 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8480b1da0 con 0x7fe8480b6c80 2020-11-20T13:56:52.752 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.747+0000 7fe846ffd700 1 --2- 172.21.15.40:0/4243316638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8480b6c80 0x7fe8480b31f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7fe83c002f30 tx=0x7fe83c006e20).ready entity=mon.0 client_cookie=f543f3a9ab583adc server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:52.752 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.747+0000 7fe845ffb700 1 -- 172.21.15.40:0/4243316638 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe83c009960 con 0x7fe8480b6c80 2020-11-20T13:56:52.752 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.747+0000 7fe845ffb700 1 -- 172.21.15.40:0/4243316638 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe83c009ac0 con 0x7fe8480b6c80 2020-11-20T13:56:52.753 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.747+0000 7fe845ffb700 1 -- 172.21.15.40:0/4243316638 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe83c009e20 con 0x7fe8480b6c80 2020-11-20T13:56:52.753 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.747+0000 7fe84d333700 1 -- 172.21.15.40:0/4243316638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8480b6c80 msgr2=0x7fe8480b31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:52.753 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.747+0000 7fe84d333700 1 --2- 172.21.15.40:0/4243316638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8480b6c80 0x7fe8480b31f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7fe83c002f30 tx=0x7fe83c006e20).stop 2020-11-20T13:56:52.753 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.747+0000 7fe84d333700 1 -- 172.21.15.40:0/4243316638 shutdown_connections 2020-11-20T13:56:52.754 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.747+0000 7fe84d333700 1 --2- 172.21.15.40:0/4243316638 >> v2:172.21.15.116:3300/0 conn(0x7fe8480b1450 0x7fe8480b18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:52.754 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.747+0000 7fe84d333700 1 --2- 172.21.15.40:0/4243316638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8480b6c80 0x7fe8480b31f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:52.754 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.747+0000 7fe84d333700 1 -- 172.21.15.40:0/4243316638 >> 172.21.15.40:0/4243316638 conn(0x7fe8480ba1d0 msgr2=0x7fe8480b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:52.754 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.747+0000 7fe84d333700 1 -- 172.21.15.40:0/4243316638 shutdown_connections 2020-11-20T13:56:52.754 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.747+0000 7fe84d333700 1 -- 172.21.15.40:0/4243316638 wait complete. 2020-11-20T13:56:52.759 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.748+0000 7fe84d333700 1 Processor -- start 2020-11-20T13:56:52.759 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.748+0000 7fe84d333700 1 -- start start 2020-11-20T13:56:52.760 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.749+0000 7fe84d333700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8480b1450 0x7fe848067000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:52.760 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.749+0000 7fe84d333700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe8480b6c80 0x7fe848157880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:52.760 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.749+0000 7fe84d333700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe848144bd0 0x7fe8481279c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:52.761 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.749+0000 7fe84d333700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe848127330 con 0x7fe8480b1450 2020-11-20T13:56:52.761 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.749+0000 7fe84d333700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8480afee0 con 0x7fe848144bd0 2020-11-20T13:56:52.761 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.749+0000 7fe84d333700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe848127520 con 0x7fe8480b6c80 2020-11-20T13:56:52.762 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.749+0000 7fe846ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8480b1450 0x7fe848067000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:52.762 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.749+0000 7fe846ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8480b1450 0x7fe848067000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55538/0 (socket says 172.21.15.40:55538) 2020-11-20T13:56:52.763 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.749+0000 7fe846ffd700 1 -- 172.21.15.40:0/782266690 learned_addr learned my addr 172.21.15.40:0/782266690 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:52.763 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.749+0000 7fe8477fe700 1 --2- 172.21.15.40:0/782266690 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe848144bd0 0x7fe8481279c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:52.763 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.749+0000 7fe8467fc700 1 --2- 172.21.15.40:0/782266690 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe8480b6c80 0x7fe848157880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:52.764 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.750+0000 7fe846ffd700 1 -- 172.21.15.40:0/782266690 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe8480b6c80 msgr2=0x7fe848157880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:52.764 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.750+0000 7fe846ffd700 1 --2- 172.21.15.40:0/782266690 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe8480b6c80 0x7fe848157880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:52.765 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.750+0000 7fe846ffd700 1 -- 172.21.15.40:0/782266690 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe848144bd0 msgr2=0x7fe8481279c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:52.765 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.750+0000 7fe846ffd700 1 --2- 172.21.15.40:0/782266690 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe848144bd0 0x7fe8481279c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:52.765 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.750+0000 7fe846ffd700 1 -- 172.21.15.40:0/782266690 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe848157d70 con 0x7fe8480b1450 2020-11-20T13:56:52.766 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.750+0000 7fe846ffd700 1 --2- 172.21.15.40:0/782266690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8480b1450 0x7fe848067000 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7fe83c002f30 tx=0x7fe83c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:52.766 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.750+0000 7fe837fff700 1 -- 172.21.15.40:0/782266690 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe83c011bd0 con 0x7fe8480b1450 2020-11-20T13:56:52.767 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.750+0000 7fe837fff700 1 -- 172.21.15.40:0/782266690 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe83c011d30 con 0x7fe8480b1450 2020-11-20T13:56:52.767 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.750+0000 7fe84d333700 1 -- 172.21.15.40:0/782266690 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8480af960 con 0x7fe8480b1450 2020-11-20T13:56:52.767 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.750+0000 7fe837fff700 1 -- 172.21.15.40:0/782266690 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe83c011ea0 con 0x7fe8480b1450 2020-11-20T13:56:52.768 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.750+0000 7fe84d333700 1 -- 172.21.15.40:0/782266690 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe848127eb0 con 0x7fe8480b1450 2020-11-20T13:56:52.768 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.751+0000 7fe84d333700 1 -- 172.21.15.40:0/782266690 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8481276e0 con 0x7fe8480b1450 2020-11-20T13:56:52.769 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.751+0000 7fe837fff700 1 -- 172.21.15.40:0/782266690 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fe83c01a040 con 0x7fe8480b1450 2020-11-20T13:56:52.769 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.751+0000 7fe837fff700 1 --2- 172.21.15.40:0/782266690 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fe8240513c0 0x7fe824053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:52.770 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.751+0000 7fe837fff700 1 -- 172.21.15.40:0/782266690 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fe83c06a080 con 0x7fe8480b1450 2020-11-20T13:56:52.770 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.751+0000 7fe8467fc700 1 --2- 172.21.15.40:0/782266690 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fe8240513c0 0x7fe824053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:52.770 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.752+0000 7fe8467fc700 1 --2- 172.21.15.40:0/782266690 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fe8240513c0 0x7fe824053880 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7fe83800d000 tx=0x7fe838007ad0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:52.771 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.755+0000 7fe837fff700 1 -- 172.21.15.40:0/782266690 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fe83c0422a0 con 0x7fe8480b1450 2020-11-20T13:56:52.917 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.915+0000 7fe84d333700 1 -- 172.21.15.40:0/782266690 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe848057050 con 0x7fe8480b1450 2020-11-20T13:56:52.917 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.915+0000 7fe837fff700 1 -- 172.21.15.40:0/782266690 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe83c023070 con 0x7fe8480b1450 2020-11-20T13:56:52.918 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2020-11-20T13:56:52.921 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.917+0000 7fe84d333700 1 -- 172.21.15.40:0/782266690 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fe8240513c0 msgr2=0x7fe824053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:52.922 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.917+0000 7fe84d333700 1 --2- 172.21.15.40:0/782266690 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fe8240513c0 0x7fe824053880 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7fe83800d000 tx=0x7fe838007ad0).stop 2020-11-20T13:56:52.922 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.917+0000 7fe84d333700 1 -- 172.21.15.40:0/782266690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8480b1450 msgr2=0x7fe848067000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:52.922 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.917+0000 7fe84d333700 1 --2- 172.21.15.40:0/782266690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8480b1450 0x7fe848067000 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7fe83c002f30 tx=0x7fe83c015000).stop 2020-11-20T13:56:52.923 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.917+0000 7fe84d333700 1 -- 172.21.15.40:0/782266690 shutdown_connections 2020-11-20T13:56:52.923 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.917+0000 7fe84d333700 1 --2- 172.21.15.40:0/782266690 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fe8240513c0 0x7fe824053880 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:52.924 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.917+0000 7fe84d333700 1 --2- 172.21.15.40:0/782266690 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe848144bd0 0x7fe8481279c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:52.924 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.917+0000 7fe84d333700 1 --2- 172.21.15.40:0/782266690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8480b1450 0x7fe848067000 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:52.924 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.917+0000 7fe84d333700 1 --2- 172.21.15.40:0/782266690 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe8480b6c80 0x7fe848157880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:52.925 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.917+0000 7fe84d333700 1 -- 172.21.15.40:0/782266690 >> 172.21.15.40:0/782266690 conn(0x7fe8480ba1d0 msgr2=0x7fe8480b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:52.925 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.917+0000 7fe84d333700 1 -- 172.21.15.40:0/782266690 shutdown_connections 2020-11-20T13:56:52.925 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:52.917+0000 7fe84d333700 1 -- 172.21.15.40:0/782266690 wait complete. 2020-11-20T13:56:52.929 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-20T13:56:53.052 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.046+0000 7f3f39f2b700 1 Processor -- start 2020-11-20T13:56:53.053 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.046+0000 7f3f39f2b700 1 -- start start 2020-11-20T13:56:53.053 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.046+0000 7f3f39f2b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3f340b6c80 0x7f3f340b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:53.053 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.046+0000 7f3f39f2b700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3f340b1450 0x7f3f340b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:53.053 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.047+0000 7f3f39f2b700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f340b0d10 con 0x7f3f34144bd0 2020-11-20T13:56:53.053 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.047+0000 7f3f39f2b700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f34145060 con 0x7f3f340b6c80 2020-11-20T13:56:53.054 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.047+0000 7f3f39f2b700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f340b0fb0 con 0x7f3f340b1450 2020-11-20T13:56:53.054 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.047+0000 7f3f337fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3f340b6c80 0x7f3f340b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:53.055 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.047+0000 7f3f337fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3f340b6c80 0x7f3f340b31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55548/0 (socket says 172.21.15.40:55548) 2020-11-20T13:56:53.055 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.047+0000 7f3f337fe700 1 -- 172.21.15.40:0/4620069 learned_addr learned my addr 172.21.15.40:0/4620069 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:53.056 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.047+0000 7f3f33fff700 1 --2- 172.21.15.40:0/4620069 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3f340b1450 0x7f3f340b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:53.056 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.047+0000 7f3f337fe700 1 -- 172.21.15.40:0/4620069 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3f340b1450 msgr2=0x7f3f340b18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:53.056 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.047+0000 7f3f337fe700 1 --2- 172.21.15.40:0/4620069 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3f340b1450 0x7f3f340b18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.056 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.047+0000 7f3f337fe700 1 -- 172.21.15.40:0/4620069 >> v1:172.21.15.116:6789/0 conn(0x7f3f34144bd0 legacy=0x7f3f340b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:53.057 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.047+0000 7f3f337fe700 1 -- 172.21.15.40:0/4620069 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f340b1da0 con 0x7f3f340b6c80 2020-11-20T13:56:53.057 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.047+0000 7f3f337fe700 1 --2- 172.21.15.40:0/4620069 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f340b6c80 0x7f3f340b31f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f3f24002f30 tx=0x7f3f24006e20).ready entity=mon.0 client_cookie=597aa6e420dc8def server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:53.057 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.047+0000 7f3f327fc700 1 -- 172.21.15.40:0/4620069 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f24009910 con 0x7f3f340b6c80 2020-11-20T13:56:53.057 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.047+0000 7f3f327fc700 1 -- 172.21.15.40:0/4620069 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f24009a70 con 0x7f3f340b6c80 2020-11-20T13:56:53.057 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.047+0000 7f3f327fc700 1 -- 172.21.15.40:0/4620069 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f24009dd0 con 0x7f3f340b6c80 2020-11-20T13:56:53.058 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.047+0000 7f3f39f2b700 1 -- 172.21.15.40:0/4620069 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f340b6c80 msgr2=0x7f3f340b31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:53.058 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.047+0000 7f3f39f2b700 1 --2- 172.21.15.40:0/4620069 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f340b6c80 0x7f3f340b31f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f3f24002f30 tx=0x7f3f24006e20).stop 2020-11-20T13:56:53.058 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.048+0000 7f3f39f2b700 1 -- 172.21.15.40:0/4620069 shutdown_connections 2020-11-20T13:56:53.058 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.048+0000 7f3f39f2b700 1 --2- 172.21.15.40:0/4620069 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3f340b1450 0x7f3f340b18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.059 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.048+0000 7f3f39f2b700 1 --2- 172.21.15.40:0/4620069 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f340b6c80 0x7f3f340b31f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.059 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.048+0000 7f3f39f2b700 1 -- 172.21.15.40:0/4620069 >> 172.21.15.40:0/4620069 conn(0x7f3f340ba1d0 msgr2=0x7f3f340b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:53.059 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.048+0000 7f3f39f2b700 1 -- 172.21.15.40:0/4620069 shutdown_connections 2020-11-20T13:56:53.060 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.048+0000 7f3f39f2b700 1 -- 172.21.15.40:0/4620069 wait complete. 2020-11-20T13:56:53.060 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.049+0000 7f3f39f2b700 1 Processor -- start 2020-11-20T13:56:53.060 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.049+0000 7f3f39f2b700 1 -- start start 2020-11-20T13:56:53.060 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.050+0000 7f3f39f2b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f340b1450 0x7f3f3405f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:53.060 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.050+0000 7f3f39f2b700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3f340b6c80 0x7f3f34066910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:53.061 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.050+0000 7f3f39f2b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f34144bd0 0x7f3f34148f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:53.061 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.050+0000 7f3f39f2b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f341488c0 con 0x7f3f340b1450 2020-11-20T13:56:53.061 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.050+0000 7f3f39f2b700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f34155370 con 0x7f3f34144bd0 2020-11-20T13:56:53.061 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.050+0000 7f3f39f2b700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f34148af0 con 0x7f3f340b6c80 2020-11-20T13:56:53.061 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.050+0000 7f3f337fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f340b1450 0x7f3f3405f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:53.062 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.050+0000 7f3f337fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f340b1450 0x7f3f3405f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55552/0 (socket says 172.21.15.40:55552) 2020-11-20T13:56:53.062 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.050+0000 7f3f337fe700 1 -- 172.21.15.40:0/4085456861 learned_addr learned my addr 172.21.15.40:0/4085456861 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:53.062 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.050+0000 7f3f32ffd700 1 --2- 172.21.15.40:0/4085456861 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3f340b6c80 0x7f3f34066910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:53.062 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.050+0000 7f3f33fff700 1 --2- 172.21.15.40:0/4085456861 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f34144bd0 0x7f3f34148f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:53.062 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.050+0000 7f3f337fe700 1 -- 172.21.15.40:0/4085456861 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3f340b6c80 msgr2=0x7f3f34066910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:53.063 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.050+0000 7f3f337fe700 1 --2- 172.21.15.40:0/4085456861 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3f340b6c80 0x7f3f34066910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.063 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.050+0000 7f3f337fe700 1 -- 172.21.15.40:0/4085456861 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f34144bd0 msgr2=0x7f3f34148f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:53.063 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.050+0000 7f3f337fe700 1 --2- 172.21.15.40:0/4085456861 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f34144bd0 0x7f3f34148f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.063 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.050+0000 7f3f337fe700 1 -- 172.21.15.40:0/4085456861 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f3414a460 con 0x7f3f340b1450 2020-11-20T13:56:53.063 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.050+0000 7f3f337fe700 1 --2- 172.21.15.40:0/4085456861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f340b1450 0x7f3f3405f710 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f3f24002f30 tx=0x7f3f24015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:53.064 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.050+0000 7f3f30ff9700 1 -- 172.21.15.40:0/4085456861 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f24011bd0 con 0x7f3f340b1450 2020-11-20T13:56:53.064 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.050+0000 7f3f30ff9700 1 -- 172.21.15.40:0/4085456861 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f24011d30 con 0x7f3f340b1450 2020-11-20T13:56:53.064 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.050+0000 7f3f30ff9700 1 -- 172.21.15.40:0/4085456861 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f24011ea0 con 0x7f3f340b1450 2020-11-20T13:56:53.064 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.050+0000 7f3f39f2b700 1 -- 172.21.15.40:0/4085456861 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f34149480 con 0x7f3f340b1450 2020-11-20T13:56:53.065 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.050+0000 7f3f39f2b700 1 -- 172.21.15.40:0/4085456861 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f3412d110 con 0x7f3f340b1450 2020-11-20T13:56:53.065 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.051+0000 7f3f39f2b700 1 -- 172.21.15.40:0/4085456861 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f34057050 con 0x7f3f340b1450 2020-11-20T13:56:53.066 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.052+0000 7f3f30ff9700 1 -- 172.21.15.40:0/4085456861 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f3f2400f070 con 0x7f3f340b1450 2020-11-20T13:56:53.066 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.052+0000 7f3f30ff9700 1 --2- 172.21.15.40:0/4085456861 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f3f140513c0 0x7f3f14053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:53.068 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.052+0000 7f3f30ff9700 1 -- 172.21.15.40:0/4085456861 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f3f2406aae0 con 0x7f3f340b1450 2020-11-20T13:56:53.068 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.052+0000 7f3f32ffd700 1 --2- 172.21.15.40:0/4085456861 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f3f140513c0 0x7f3f14053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:53.068 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.052+0000 7f3f32ffd700 1 --2- 172.21.15.40:0/4085456861 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f3f140513c0 0x7f3f14053880 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f3f2800d000 tx=0x7f3f28007ad0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:53.069 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.055+0000 7f3f30ff9700 1 -- 172.21.15.40:0/4085456861 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f3f24042d00 con 0x7f3f340b1450 2020-11-20T13:56:53.216 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.215+0000 7f3f39f2b700 1 -- 172.21.15.40:0/4085456861 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3f34067c20 con 0x7f3f340b1450 2020-11-20T13:56:53.217 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.216+0000 7f3f30ff9700 1 -- 172.21.15.40:0/4085456861 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+10 (secure 0 0 0) 0x7f3f24046320 con 0x7f3f340b1450 2020-11-20T13:56:53.218 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2020-11-20T13:56:53.219 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.217+0000 7f3f39f2b700 1 -- 172.21.15.40:0/4085456861 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f3f140513c0 msgr2=0x7f3f14053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:53.219 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.217+0000 7f3f39f2b700 1 --2- 172.21.15.40:0/4085456861 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f3f140513c0 0x7f3f14053880 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f3f2800d000 tx=0x7f3f28007ad0).stop 2020-11-20T13:56:53.219 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.217+0000 7f3f39f2b700 1 -- 172.21.15.40:0/4085456861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f340b1450 msgr2=0x7f3f3405f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:53.219 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.217+0000 7f3f39f2b700 1 --2- 172.21.15.40:0/4085456861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f340b1450 0x7f3f3405f710 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f3f24002f30 tx=0x7f3f24015000).stop 2020-11-20T13:56:53.220 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.217+0000 7f3f39f2b700 1 -- 172.21.15.40:0/4085456861 shutdown_connections 2020-11-20T13:56:53.220 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.217+0000 7f3f39f2b700 1 --2- 172.21.15.40:0/4085456861 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f3f140513c0 0x7f3f14053880 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.220 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.217+0000 7f3f39f2b700 1 --2- 172.21.15.40:0/4085456861 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f34144bd0 0x7f3f34148f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.220 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.217+0000 7f3f39f2b700 1 --2- 172.21.15.40:0/4085456861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f340b1450 0x7f3f3405f710 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.220 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.217+0000 7f3f39f2b700 1 --2- 172.21.15.40:0/4085456861 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3f340b6c80 0x7f3f34066910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.221 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.217+0000 7f3f39f2b700 1 -- 172.21.15.40:0/4085456861 >> 172.21.15.40:0/4085456861 conn(0x7f3f340ba1d0 msgr2=0x7f3f340afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:53.221 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.217+0000 7f3f39f2b700 1 -- 172.21.15.40:0/4085456861 shutdown_connections 2020-11-20T13:56:53.221 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.217+0000 7f3f39f2b700 1 -- 172.21.15.40:0/4085456861 wait complete. 2020-11-20T13:56:53.229 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-20T13:56:53.346 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.344+0000 7fb52d8d0700 1 Processor -- start 2020-11-20T13:56:53.346 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.344+0000 7fb52d8d0700 1 -- start start 2020-11-20T13:56:53.348 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.345+0000 7fb52d8d0700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb5280b05c0 0x7fb5280b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:53.348 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.345+0000 7fb52d8d0700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb5280b54c0 0x7fb5280ad430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:53.349 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.345+0000 7fb52d8d0700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb528144730 con 0x7fb5281480a0 2020-11-20T13:56:53.349 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.345+0000 7fb52d8d0700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5280ad040 con 0x7fb5280b05c0 2020-11-20T13:56:53.349 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.345+0000 7fb52d8d0700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5281449a0 con 0x7fb5280b54c0 2020-11-20T13:56:53.349 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.345+0000 7fb527fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb5280b05c0 0x7fb5280b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:53.350 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.345+0000 7fb52d0cf700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb5280b54c0 0x7fb5280ad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:53.350 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.345+0000 7fb52d0cf700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb5280b54c0 0x7fb5280ad430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.40:40638/0 (socket says 172.21.15.40:40638) 2020-11-20T13:56:53.350 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.346+0000 7fb52d0cf700 1 -- 172.21.15.40:0/2237334818 learned_addr learned my addr 172.21.15.40:0/2237334818 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:53.350 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.345+0000 7fb52c8ce700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fb5281480a0 0x7fb528144020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.40:58788/0 (socket says 172.21.15.40:58788) 2020-11-20T13:56:53.350 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.345+0000 7fb527fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb5280b05c0 0x7fb5280b4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.40:46360/0 (socket says 172.21.15.40:46360) 2020-11-20T13:56:53.351 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.346+0000 7fb527fff700 1 -- 172.21.15.40:0/2237334818 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb5280b54c0 msgr2=0x7fb5280ad430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:53.351 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.346+0000 7fb527fff700 1 --2- 172.21.15.40:0/2237334818 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb5280b54c0 0x7fb5280ad430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.351 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.346+0000 7fb527fff700 1 -- 172.21.15.40:0/2237334818 >> v1:172.21.15.116:6789/0 conn(0x7fb5281480a0 legacy=0x7fb528144020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:53.351 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.346+0000 7fb527fff700 1 -- 172.21.15.40:0/2237334818 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb528144c40 con 0x7fb5280b05c0 2020-11-20T13:56:53.352 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.346+0000 7fb527fff700 1 --2- 172.21.15.40:0/2237334818 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5280b05c0 0x7fb5280b4fd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fb51c002f30 tx=0x7fb51c006dc0).ready entity=mon.1 client_cookie=a699695d2945c3f6 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:53.352 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.346+0000 7fb5277fe700 1 -- 172.21.15.40:0/2237334818 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb51c009760 con 0x7fb5280b05c0 2020-11-20T13:56:53.352 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.346+0000 7fb5277fe700 1 -- 172.21.15.40:0/2237334818 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51c0098c0 con 0x7fb5280b05c0 2020-11-20T13:56:53.352 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.346+0000 7fb5277fe700 1 -- 172.21.15.40:0/2237334818 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb51c009c20 con 0x7fb5280b05c0 2020-11-20T13:56:53.352 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.346+0000 7fb52d8d0700 1 -- 172.21.15.40:0/2237334818 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5280b05c0 msgr2=0x7fb5280b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:53.353 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.346+0000 7fb52d8d0700 1 --2- 172.21.15.40:0/2237334818 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5280b05c0 0x7fb5280b4fd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fb51c002f30 tx=0x7fb51c006dc0).stop 2020-11-20T13:56:53.353 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.347+0000 7fb52d8d0700 1 -- 172.21.15.40:0/2237334818 shutdown_connections 2020-11-20T13:56:53.353 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.347+0000 7fb52d8d0700 1 --2- 172.21.15.40:0/2237334818 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb5280b54c0 0x7fb5280ad430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.353 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.347+0000 7fb52d8d0700 1 --2- 172.21.15.40:0/2237334818 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5280b05c0 0x7fb5280b4fd0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.353 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.347+0000 7fb52d8d0700 1 -- 172.21.15.40:0/2237334818 >> 172.21.15.40:0/2237334818 conn(0x7fb5280b9520 msgr2=0x7fb5280b9930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:53.354 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.347+0000 7fb52d8d0700 1 -- 172.21.15.40:0/2237334818 shutdown_connections 2020-11-20T13:56:53.354 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.347+0000 7fb52d8d0700 1 -- 172.21.15.40:0/2237334818 wait complete. 2020-11-20T13:56:53.357 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.348+0000 7fb52d8d0700 1 Processor -- start 2020-11-20T13:56:53.358 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.348+0000 7fb52d8d0700 1 -- start start 2020-11-20T13:56:53.358 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.348+0000 7fb52d8d0700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb5280b05c0 0x7fb528065fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:53.358 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.348+0000 7fb52d8d0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5280b54c0 0x7fb5281284f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:53.358 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.348+0000 7fb52d8d0700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5281480a0 0x7fb528145fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:53.358 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.348+0000 7fb52d8d0700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb528127b10 con 0x7fb5280b54c0 2020-11-20T13:56:53.359 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.348+0000 7fb52d8d0700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb528127db0 con 0x7fb5281480a0 2020-11-20T13:56:53.359 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.348+0000 7fb52d8d0700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb528128050 con 0x7fb5280b05c0 2020-11-20T13:56:53.359 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.349+0000 7fb527fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5280b54c0 0x7fb5281284f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:53.359 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.349+0000 7fb527fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5280b54c0 0x7fb5281284f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55568/0 (socket says 172.21.15.40:55568) 2020-11-20T13:56:53.360 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.349+0000 7fb527fff700 1 -- 172.21.15.40:0/1458090276 learned_addr learned my addr 172.21.15.40:0/1458090276 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:53.360 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.349+0000 7fb52c8ce700 1 --2- 172.21.15.40:0/1458090276 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb5280b05c0 0x7fb528065fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:53.360 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.349+0000 7fb52d0cf700 1 --2- 172.21.15.40:0/1458090276 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5281480a0 0x7fb528145fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:53.360 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.349+0000 7fb527fff700 1 -- 172.21.15.40:0/1458090276 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb5280b05c0 msgr2=0x7fb528065fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:53.360 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.349+0000 7fb527fff700 1 --2- 172.21.15.40:0/1458090276 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb5280b05c0 0x7fb528065fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.361 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.349+0000 7fb527fff700 1 -- 172.21.15.40:0/1458090276 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5281480a0 msgr2=0x7fb528145fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:53.361 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.349+0000 7fb527fff700 1 --2- 172.21.15.40:0/1458090276 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5281480a0 0x7fb528145fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.361 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.349+0000 7fb527fff700 1 -- 172.21.15.40:0/1458090276 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb528066490 con 0x7fb5280b54c0 2020-11-20T13:56:53.361 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.349+0000 7fb527fff700 1 --2- 172.21.15.40:0/1458090276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5280b54c0 0x7fb5281284f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7fb51c002f30 tx=0x7fb51c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:53.361 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.349+0000 7fb525ffb700 1 -- 172.21.15.40:0/1458090276 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb51c011910 con 0x7fb5280b54c0 2020-11-20T13:56:53.362 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.349+0000 7fb525ffb700 1 -- 172.21.15.40:0/1458090276 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51c011a70 con 0x7fb5280b54c0 2020-11-20T13:56:53.362 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.349+0000 7fb52d8d0700 1 -- 172.21.15.40:0/1458090276 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5281289e0 con 0x7fb5280b54c0 2020-11-20T13:56:53.362 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.349+0000 7fb525ffb700 1 -- 172.21.15.40:0/1458090276 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb51c011dd0 con 0x7fb5280b54c0 2020-11-20T13:56:53.364 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.349+0000 7fb52d8d0700 1 -- 172.21.15.40:0/1458090276 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb528065c20 con 0x7fb5280b54c0 2020-11-20T13:56:53.364 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.350+0000 7fb52d8d0700 1 -- 172.21.15.40:0/1458090276 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb528057050 con 0x7fb5280b54c0 2020-11-20T13:56:53.364 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.350+0000 7fb525ffb700 1 -- 172.21.15.40:0/1458090276 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fb51c024030 con 0x7fb5280b54c0 2020-11-20T13:56:53.364 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.350+0000 7fb525ffb700 1 --2- 172.21.15.40:0/1458090276 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fb5080512d0 0x7fb508053790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:53.364 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.351+0000 7fb525ffb700 1 -- 172.21.15.40:0/1458090276 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fb51c069e30 con 0x7fb5280b54c0 2020-11-20T13:56:53.365 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.351+0000 7fb52c8ce700 1 --2- 172.21.15.40:0/1458090276 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fb5080512d0 0x7fb508053790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:53.365 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.351+0000 7fb52c8ce700 1 --2- 172.21.15.40:0/1458090276 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fb5080512d0 0x7fb508053790 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fb51800d000 tx=0x7fb518007ac0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:53.365 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.354+0000 7fb525ffb700 1 -- 172.21.15.40:0/1458090276 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fb51c0420d0 con 0x7fb5280b54c0 2020-11-20T13:56:53.516 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.515+0000 7fb52d8d0700 1 -- 172.21.15.40:0/1458090276 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb5281434f0 con 0x7fb5280b54c0 2020-11-20T13:56:53.516 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.515+0000 7fb525ffb700 1 -- 172.21.15.40:0/1458090276 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+10 (secure 0 0 0) 0x7fb51c022020 con 0x7fb5280b54c0 2020-11-20T13:56:53.517 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2020-11-20T13:56:53.530 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.516+0000 7fb52d8d0700 1 -- 172.21.15.40:0/1458090276 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fb5080512d0 msgr2=0x7fb508053790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:53.530 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.516+0000 7fb52d8d0700 1 --2- 172.21.15.40:0/1458090276 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fb5080512d0 0x7fb508053790 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fb51800d000 tx=0x7fb518007ac0).stop 2020-11-20T13:56:53.530 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.516+0000 7fb52d8d0700 1 -- 172.21.15.40:0/1458090276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5280b54c0 msgr2=0x7fb5281284f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:53.531 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.516+0000 7fb52d8d0700 1 --2- 172.21.15.40:0/1458090276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5280b54c0 0x7fb5281284f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7fb51c002f30 tx=0x7fb51c015000).stop 2020-11-20T13:56:53.531 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.516+0000 7fb52d8d0700 1 -- 172.21.15.40:0/1458090276 shutdown_connections 2020-11-20T13:56:53.531 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.516+0000 7fb52d8d0700 1 --2- 172.21.15.40:0/1458090276 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fb5080512d0 0x7fb508053790 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.533 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.516+0000 7fb52d8d0700 1 --2- 172.21.15.40:0/1458090276 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb5280b05c0 0x7fb528065fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.533 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.516+0000 7fb52d8d0700 1 --2- 172.21.15.40:0/1458090276 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5281480a0 0x7fb528145fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.533 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.516+0000 7fb52d8d0700 1 --2- 172.21.15.40:0/1458090276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5280b54c0 0x7fb5281284f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.533 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.516+0000 7fb52d8d0700 1 -- 172.21.15.40:0/1458090276 >> 172.21.15.40:0/1458090276 conn(0x7fb5280b9520 msgr2=0x7fb52812cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:53.534 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.516+0000 7fb52d8d0700 1 -- 172.21.15.40:0/1458090276 shutdown_connections 2020-11-20T13:56:53.534 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.516+0000 7fb52d8d0700 1 -- 172.21.15.40:0/1458090276 wait complete. 2020-11-20T13:56:53.534 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-20T13:56:53.658 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.656+0000 7f849cdfe700 1 Processor -- start 2020-11-20T13:56:53.658 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.657+0000 7f849cdfe700 1 -- start start 2020-11-20T13:56:53.660 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.657+0000 7f849cdfe700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f849809bc10 0x7f84980aff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:53.661 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.657+0000 7f849cdfe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f84980b5260 0x7f84980b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:53.661 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.657+0000 7f849cdfe700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f849809bff0 con 0x7f84980b81c0 2020-11-20T13:56:53.662 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.657+0000 7f849cdfe700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84980b8740 con 0x7f84980b5260 2020-11-20T13:56:53.662 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.657+0000 7f849cdfe700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84980b89e0 con 0x7f849809bc10 2020-11-20T13:56:53.662 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.658+0000 7f849659c700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f849809bc10 0x7f84980aff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:53.662 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.658+0000 7f8496d9d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f84980b5260 0x7f84980b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:53.663 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.658+0000 7f849659c700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f849809bc10 0x7f84980aff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.40:40648/0 (socket says 172.21.15.40:40648) 2020-11-20T13:56:53.663 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.658+0000 7f8495d9b700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f84980b81c0 0x7f8498099f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.40:58802/0 (socket says 172.21.15.40:58802) 2020-11-20T13:56:53.663 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.658+0000 7f849659c700 1 -- 172.21.15.40:0/3809569042 learned_addr learned my addr 172.21.15.40:0/3809569042 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:53.663 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.658+0000 7f8496d9d700 1 -- 172.21.15.40:0/3809569042 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f849809bc10 msgr2=0x7f84980aff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:53.664 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.658+0000 7f8496d9d700 1 --2- 172.21.15.40:0/3809569042 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f849809bc10 0x7f84980aff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.664 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.658+0000 7f8496d9d700 1 -- 172.21.15.40:0/3809569042 >> v1:172.21.15.116:6789/0 conn(0x7f84980b81c0 legacy=0x7f8498099f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:53.664 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.658+0000 7f8496d9d700 1 -- 172.21.15.40:0/3809569042 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84980b8c80 con 0x7f84980b5260 2020-11-20T13:56:53.664 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.658+0000 7f849659c700 1 --2- 172.21.15.40:0/3809569042 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f849809bc10 0x7f84980aff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:56:53.664 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.658+0000 7f8496d9d700 1 --2- 172.21.15.40:0/3809569042 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84980b5260 0x7f84980b0430 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f84900032b0 tx=0x7f84900091a0).ready entity=mon.1 client_cookie=874f1ca518890d08 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:53.665 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.658+0000 7f849559a700 1 -- 172.21.15.40:0/3809569042 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f849000b9a0 con 0x7f84980b5260 2020-11-20T13:56:53.665 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.658+0000 7f849559a700 1 -- 172.21.15.40:0/3809569042 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f849000bb00 con 0x7f84980b5260 2020-11-20T13:56:53.665 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.658+0000 7f849559a700 1 -- 172.21.15.40:0/3809569042 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f849000bc70 con 0x7f84980b5260 2020-11-20T13:56:53.665 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.658+0000 7f849cdfe700 1 -- 172.21.15.40:0/3809569042 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84980b5260 msgr2=0x7f84980b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:53.665 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.658+0000 7f849cdfe700 1 --2- 172.21.15.40:0/3809569042 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84980b5260 0x7f84980b0430 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f84900032b0 tx=0x7f84900091a0).stop 2020-11-20T13:56:53.666 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.659+0000 7f849cdfe700 1 -- 172.21.15.40:0/3809569042 shutdown_connections 2020-11-20T13:56:53.666 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.659+0000 7f849cdfe700 1 --2- 172.21.15.40:0/3809569042 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84980b5260 0x7f84980b0430 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.666 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.659+0000 7f849cdfe700 1 --2- 172.21.15.40:0/3809569042 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f849809bc10 0x7f84980aff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.681 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.659+0000 7f849cdfe700 1 -- 172.21.15.40:0/3809569042 >> 172.21.15.40:0/3809569042 conn(0x7f84980ba380 msgr2=0x7f84980b6350 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:53.682 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.659+0000 7f849cdfe700 1 -- 172.21.15.40:0/3809569042 shutdown_connections 2020-11-20T13:56:53.682 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.659+0000 7f849cdfe700 1 -- 172.21.15.40:0/3809569042 wait complete. 2020-11-20T13:56:53.684 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.660+0000 7f849cdfe700 1 Processor -- start 2020-11-20T13:56:53.684 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.660+0000 7f849cdfe700 1 -- start start 2020-11-20T13:56:53.685 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.661+0000 7f849cdfe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f849809bc10 0x7f849813b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:53.685 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.661+0000 7f849cdfe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84980b5260 0x7f84981440d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:53.685 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.661+0000 7f849cdfe700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f84980b81c0 0x7f849809da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:53.685 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.661+0000 7f849cdfe700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849809f120 con 0x7f84980b5260 2020-11-20T13:56:53.686 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.661+0000 7f849cdfe700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8498063e10 con 0x7f849809bc10 2020-11-20T13:56:53.686 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.661+0000 7f849cdfe700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84981457b0 con 0x7f84980b81c0 2020-11-20T13:56:53.686 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.661+0000 7f8495d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84980b5260 0x7f84981440d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:53.686 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.661+0000 7f8495d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84980b5260 0x7f84981440d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55582/0 (socket says 172.21.15.40:55582) 2020-11-20T13:56:53.686 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.661+0000 7f8495d9b700 1 -- 172.21.15.40:0/2059312670 learned_addr learned my addr 172.21.15.40:0/2059312670 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:53.687 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.661+0000 7f849659c700 1 --2- 172.21.15.40:0/2059312670 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f849809bc10 0x7f849813b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:53.687 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.661+0000 7f8496d9d700 1 --2- 172.21.15.40:0/2059312670 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f84980b81c0 0x7f849809da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:53.687 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.661+0000 7f8495d9b700 1 -- 172.21.15.40:0/2059312670 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f84980b81c0 msgr2=0x7f849809da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:53.687 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.661+0000 7f8495d9b700 1 --2- 172.21.15.40:0/2059312670 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f84980b81c0 0x7f849809da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.687 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.661+0000 7f8495d9b700 1 -- 172.21.15.40:0/2059312670 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f849809bc10 msgr2=0x7f849813b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:53.688 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.661+0000 7f8495d9b700 1 --2- 172.21.15.40:0/2059312670 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f849809bc10 0x7f849813b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.688 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.661+0000 7f8495d9b700 1 -- 172.21.15.40:0/2059312670 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8498063b00 con 0x7f84980b5260 2020-11-20T13:56:53.688 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.661+0000 7f8495d9b700 1 --2- 172.21.15.40:0/2059312670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84980b5260 0x7f84981440d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f84980b9000 tx=0x7f8488007bd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:53.688 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.661+0000 7f848f7fe700 1 -- 172.21.15.40:0/2059312670 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84880072e0 con 0x7f84980b5260 2020-11-20T13:56:53.688 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.661+0000 7f848f7fe700 1 -- 172.21.15.40:0/2059312670 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8488005520 con 0x7f84980b5260 2020-11-20T13:56:53.689 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.661+0000 7f849cdfe700 1 -- 172.21.15.40:0/2059312670 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f849809ec10 con 0x7f84980b5260 2020-11-20T13:56:53.689 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.661+0000 7f848f7fe700 1 -- 172.21.15.40:0/2059312670 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8488005880 con 0x7f84980b5260 2020-11-20T13:56:53.689 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.661+0000 7f849cdfe700 1 -- 172.21.15.40:0/2059312670 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8498144780 con 0x7f84980b5260 2020-11-20T13:56:53.689 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.663+0000 7f848f7fe700 1 -- 172.21.15.40:0/2059312670 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f848801a060 con 0x7f84980b5260 2020-11-20T13:56:53.689 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.663+0000 7f849cdfe700 1 -- 172.21.15.40:0/2059312670 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f849809ef70 con 0x7f84980b5260 2020-11-20T13:56:53.690 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.663+0000 7f848f7fe700 1 --2- 172.21.15.40:0/2059312670 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f847c051210 0x7f847c0536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:53.690 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.663+0000 7f848f7fe700 1 -- 172.21.15.40:0/2059312670 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f8488068790 con 0x7f84980b5260 2020-11-20T13:56:53.690 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.663+0000 7f849659c700 1 --2- 172.21.15.40:0/2059312670 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f847c051210 0x7f847c0536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:53.690 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.663+0000 7f849659c700 1 --2- 172.21.15.40:0/2059312670 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f847c051210 0x7f847c0536d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f8480008000 tx=0x7f8480008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:53.690 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.666+0000 7f848f7fe700 1 -- 172.21.15.40:0/2059312670 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f84880409b0 con 0x7f84980b5260 2020-11-20T13:56:53.827 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.826+0000 7f849cdfe700 1 -- 172.21.15.40:0/2059312670 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8498057050 con 0x7f84980b5260 2020-11-20T13:56:53.828 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.827+0000 7f848f7fe700 1 -- 172.21.15.40:0/2059312670 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16347 (secure 0 0 0) 0x7f8488018020 con 0x7f84980b5260 2020-11-20T13:56:53.828 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:56:53.828 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":24,"fsid":"da1ed737-5174-430f-a3c9-56f6bbfad13c","created":"2020-11-20T13:56:04.990651+0000","modified":"2020-11-20T13:56:25.428032+0000","last_up_change":"2020-11-20T13:56:13.893026+0000","last_in_change":"2020-11-20T13:56:13.893026+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T13:56:08.330017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T13:56:17.939684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T13:56:20.563740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T13:56:23.062623+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9ea4c667-8938-46d6-a5ca-092d55c1f79e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":22825},{"type":"v1","addr":"172.21.15.40:6818","nonce":22825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":22825},{"type":"v1","addr":"172.21.15.40:6822","nonce":22825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":22825},{"type":"v1","addr":"172.21.15.40:6830","nonce":22825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":22825},{"type":"v1","addr":"172.21.15.40:6826","nonce":22825}]},"public_addr":"172.21.15.40:6818/22825","cluster_addr":"172.21.15.40:6822/22825","heartbeat_back_addr":"172.21.15.40:6830/22825","heartbeat_front_addr":"172.21.15.40:6826/22825","state":["exists","up"]},{"osd":1,"uuid":"515708d9-5087-4e55-8d14-cdea1c00eacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":22823},{"type":"v1","addr":"172.21.15.40:6801","nonce":22823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":22823},{"type":"v1","addr":"172.21.15.40:6803","nonce":22823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":22823},{"type":"v1","addr":"172.21.15.40:6807","nonce":22823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":22823},{"type":"v1","addr":"172.21.15.40:6805","nonce":22823}]},"public_addr":"172.21.15.40:6801/22823","cluster_addr":"172.21.15.40:6803/22823","heartbeat_back_addr":"172.21.15.40:6807/22823","heartbeat_front_addr":"172.21.15.40:6805/22823","state":["exists","up"]},{"osd":2,"uuid":"4d86bca7-743e-4983-b35f-36676c405b36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":22824},{"type":"v1","addr":"172.21.15.40:6809","nonce":22824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":22824},{"type":"v1","addr":"172.21.15.40:6811","nonce":22824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":22824},{"type":"v1","addr":"172.21.15.40:6815","nonce":22824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":22824},{"type":"v1","addr":"172.21.15.40:6813","nonce":22824}]},"public_addr":"172.21.15.40:6809/22824","cluster_addr":"172.21.15.40:6811/22824","heartbeat_back_addr":"172.21.15.40:6815/22824","heartbeat_front_addr":"172.21.15.40:6813/22824","state":["exists","up"]},{"osd":3,"uuid":"f4dc726b-6b34-4e8b-8eeb-87060b58002d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":22826},{"type":"v1","addr":"172.21.15.40:6819","nonce":22826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":22826},{"type":"v1","addr":"172.21.15.40:6823","nonce":22826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":22826},{"type":"v1","addr":"172.21.15.40:6831","nonce":22826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":22826},{"type":"v1","addr":"172.21.15.40:6827","nonce":22826}]},"public_addr":"172.21.15.40:6819/22826","cluster_addr":"172.21.15.40:6823/22826","heartbeat_back_addr":"172.21.15.40:6831/22826","heartbeat_front_addr":"172.21.15.40:6827/22826","state":["exists","up"]},{"osd":4,"uuid":"25a6d53b-d1f0-4498-b7d5-ffa38661d403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":23181},{"type":"v1","addr":"172.21.15.116:6819","nonce":23181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":23181},{"type":"v1","addr":"172.21.15.116:6821","nonce":23181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":23181},{"type":"v1","addr":"172.21.15.116:6825","nonce":23181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":23181},{"type":"v1","addr":"172.21.15.116:6823","nonce":23181}]},"public_addr":"172.21.15.116:6819/23181","cluster_addr":"172.21.15.116:6821/23181","heartbeat_back_addr":"172.21.15.116:6825/23181","heartbeat_front_addr":"172.21.15.116:6823/23181","state":["exists","up"]},{"osd":5,"uuid":"7d7a5af8-229b-4351-9a21-a3d509cd4635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":23171},{"type":"v1","addr":"172.21.15.116:6803","nonce":23171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":23171},{"type":"v1","addr":"172.21.15.116:6805","nonce":23171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":23171},{"type":"v1","addr":"172.21.15.116:6809","nonce":23171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":23171},{"type":"v1","addr":"172.21.15.116:6807","nonce":23171}]},"public_addr":"172.21.15.116:6803/23171","cluster_addr":"172.21.15.116:6805/23171","heartbeat_back_addr":"172.21.15.116:6809/23171","heartbeat_front_addr":"172.21.15.116:6807/23171","state":["exists","up"]},{"osd":6,"uuid":"8247d248-a140-4e36-9796-9847a97df58d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":23172},{"type":"v1","addr":"172.21.15.116:6811","nonce":23172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":23172},{"type":"v1","addr":"172.21.15.116:6813","nonce":23172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":23172},{"type":"v1","addr":"172.21.15.116:6817","nonce":23172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":23172},{"type":"v1","addr":"172.21.15.116:6815","nonce":23172}]},"public_addr":"172.21.15.116:6811/23172","cluster_addr":"172.21.15.116:6813/23172","heartbeat_back_addr":"172.21.15.116:6817/23172","heartbeat_front_addr":"172.21.15.116:6815/23172","state":["exists","up"]},{"osd":7,"uuid":"472803c4-70b8-499d-b18a-71aaeec6aab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":23196},{"type":"v1","addr":"172.21.15.116:6827","nonce":23196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":23196},{"type":"v1","addr":"172.21.15.116:6829","nonce":23196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":23196},{"type":"v1","addr":"172.21.15.116:6833","nonce":23196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":23196},{"type":"v1","addr":"172.21.15.116:6831","nonce":23196}]},"public_addr":"172.21.15.116:6827/23196","cluster_addr":"172.21.15.116:6829/23196","heartbeat_back_addr":"172.21.15.116:6833/23196","heartbeat_front_addr":"172.21.15.116:6831/23196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.029357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:11.989797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.091097+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.014766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.070860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.071864+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.063400+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.109028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-20T13:56:53.841 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.828+0000 7f849cdfe700 1 -- 172.21.15.40:0/2059312670 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f847c051210 msgr2=0x7f847c0536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:53.841 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.828+0000 7f849cdfe700 1 --2- 172.21.15.40:0/2059312670 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f847c051210 0x7f847c0536d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f8480008000 tx=0x7f8480008040).stop 2020-11-20T13:56:53.842 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.828+0000 7f849cdfe700 1 -- 172.21.15.40:0/2059312670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84980b5260 msgr2=0x7f84981440d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:53.842 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.828+0000 7f849cdfe700 1 --2- 172.21.15.40:0/2059312670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84980b5260 0x7f84981440d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f84980b9000 tx=0x7f8488007bd0).stop 2020-11-20T13:56:53.842 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.828+0000 7f849cdfe700 1 -- 172.21.15.40:0/2059312670 shutdown_connections 2020-11-20T13:56:53.842 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.829+0000 7f849cdfe700 1 --2- 172.21.15.40:0/2059312670 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f847c051210 0x7f847c0536d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.843 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.829+0000 7f849cdfe700 1 --2- 172.21.15.40:0/2059312670 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f84980b81c0 0x7f849809da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.843 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.829+0000 7f849cdfe700 1 --2- 172.21.15.40:0/2059312670 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f849809bc10 0x7f849813b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.843 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.829+0000 7f849cdfe700 1 --2- 172.21.15.40:0/2059312670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84980b5260 0x7f84981440d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.843 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.829+0000 7f849cdfe700 1 -- 172.21.15.40:0/2059312670 >> 172.21.15.40:0/2059312670 conn(0x7f84980ba380 msgr2=0x7f849805f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:53.843 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.829+0000 7f849cdfe700 1 -- 172.21.15.40:0/2059312670 shutdown_connections 2020-11-20T13:56:53.844 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.829+0000 7f849cdfe700 1 -- 172.21.15.40:0/2059312670 wait complete. 2020-11-20T13:56:53.844 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T13:56:08.330017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T13:56:17.939684+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T13:56:20.563740+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T13:56:23.062623+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T13:56:53.845 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-20T13:56:53.962 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.961+0000 7fefafd39700 1 Processor -- start 2020-11-20T13:56:53.962 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.961+0000 7fefafd39700 1 -- start start 2020-11-20T13:56:53.964 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.961+0000 7fefafd39700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fefa80b5bf0 0x7fefa805f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:53.965 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.962+0000 7fefafd39700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefa80b5fd0 con 0x7fefa8144390 2020-11-20T13:56:53.965 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.962+0000 7fefafd39700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefa80b05f0 con 0x7fefa80b3960 2020-11-20T13:56:53.965 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.962+0000 7fefafd39700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefa8144870 con 0x7fefa80b5bf0 2020-11-20T13:56:53.965 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.962+0000 7fefadad5700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fefa80b5bf0 0x7fefa805f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:53.966 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.962+0000 7fefad2d4700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fefa8144390 0x7fefa805f880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46184/0 (socket says 172.21.15.40:46184) 2020-11-20T13:56:53.966 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.962+0000 7fefad2d4700 1 -- 172.21.15.40:0/1690359096 learned_addr learned my addr 172.21.15.40:0/1690359096 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:53.966 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.962+0000 7fefadad5700 1 --2- 172.21.15.40:0/1690359096 >> v2:172.21.15.40:3300/0 conn(0x7fefa80b5bf0 0x7fefa805f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55588/0 (socket says 172.21.15.40:55588) 2020-11-20T13:56:53.966 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.962+0000 7fefadad5700 1 -- 172.21.15.40:0/1690359096 >> v1:172.21.15.116:6789/0 conn(0x7fefa80b3960 legacy=0x7fefa814cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:53.966 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.962+0000 7fefadad5700 1 -- 172.21.15.40:0/1690359096 >> v1:172.21.15.40:6789/0 conn(0x7fefa8144390 legacy=0x7fefa805f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:53.967 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.962+0000 7fefadad5700 1 -- 172.21.15.40:0/1690359096 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa80b0c30 con 0x7fefa80b5bf0 2020-11-20T13:56:53.967 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.962+0000 7fefacad3700 1 -- 172.21.15.40:0/1690359096 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3958346528 0 0) 0x7fefa80b5fd0 con 0x7fefa8144390 2020-11-20T13:56:53.967 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.962+0000 7fefadad5700 1 --2- 172.21.15.40:0/1690359096 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefa80b5bf0 0x7fefa805f390 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7fefa00032b0 tx=0x7fefa000b3e0).ready entity=mon.0 client_cookie=8b517c29f819bc97 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:53.967 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.962+0000 7fefacad3700 1 -- 172.21.15.40:0/1690359096 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefa0010660 con 0x7fefa80b5bf0 2020-11-20T13:56:53.967 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.962+0000 7fefacad3700 1 -- 172.21.15.40:0/1690359096 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa00107c0 con 0x7fefa80b5bf0 2020-11-20T13:56:53.968 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.962+0000 7fefacad3700 1 -- 172.21.15.40:0/1690359096 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefa0010b20 con 0x7fefa80b5bf0 2020-11-20T13:56:53.968 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.962+0000 7fefafd39700 1 -- 172.21.15.40:0/1690359096 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefa80b5bf0 msgr2=0x7fefa805f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:53.968 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.962+0000 7fefafd39700 1 --2- 172.21.15.40:0/1690359096 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefa80b5bf0 0x7fefa805f390 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7fefa00032b0 tx=0x7fefa000b3e0).stop 2020-11-20T13:56:53.968 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.963+0000 7fefafd39700 1 -- 172.21.15.40:0/1690359096 shutdown_connections 2020-11-20T13:56:53.968 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.963+0000 7fefafd39700 1 --2- 172.21.15.40:0/1690359096 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefa80b5bf0 0x7fefa805f390 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.968 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.963+0000 7fefafd39700 1 -- 172.21.15.40:0/1690359096 >> 172.21.15.40:0/1690359096 conn(0x7fefa80ba1d0 msgr2=0x7fefa80b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:53.970 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.963+0000 7fefafd39700 1 -- 172.21.15.40:0/1690359096 shutdown_connections 2020-11-20T13:56:53.970 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.963+0000 7fefafd39700 1 -- 172.21.15.40:0/1690359096 wait complete. 2020-11-20T13:56:53.972 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.964+0000 7fefafd39700 1 Processor -- start 2020-11-20T13:56:53.972 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.964+0000 7fefafd39700 1 -- start start 2020-11-20T13:56:53.972 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.965+0000 7fefafd39700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fefa80b3960 0x7fefa8152f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:53.973 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.965+0000 7fefafd39700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fefa80b5bf0 0x7fefa813ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:53.973 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.965+0000 7fefafd39700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefa8144390 0x7fefa80ad1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:53.973 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.965+0000 7fefafd39700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa80ad6c0 con 0x7fefa8144390 2020-11-20T13:56:53.973 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.965+0000 7fefafd39700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa80adc20 con 0x7fefa80b5bf0 2020-11-20T13:56:53.974 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.965+0000 7fefafd39700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefa80adec0 con 0x7fefa80b3960 2020-11-20T13:56:53.974 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.965+0000 7fefae2d6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefa8144390 0x7fefa80ad1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:53.974 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.965+0000 7fefae2d6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefa8144390 0x7fefa80ad1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55598/0 (socket says 172.21.15.40:55598) 2020-11-20T13:56:53.974 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.965+0000 7fefae2d6700 1 -- 172.21.15.40:0/1722994594 learned_addr learned my addr 172.21.15.40:0/1722994594 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:53.974 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.965+0000 7fefadad5700 1 --2- 172.21.15.40:0/1722994594 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fefa80b3960 0x7fefa8152f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:53.975 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.965+0000 7fefae2d6700 1 -- 172.21.15.40:0/1722994594 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fefa80b3960 msgr2=0x7fefa8152f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:53.975 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.965+0000 7fefae2d6700 1 --2- 172.21.15.40:0/1722994594 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fefa80b3960 0x7fefa8152f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.975 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.965+0000 7fefae2d6700 1 -- 172.21.15.40:0/1722994594 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fefa80b5bf0 msgr2=0x7fefa813ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:53.975 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.965+0000 7fefae2d6700 1 --2- 172.21.15.40:0/1722994594 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fefa80b5bf0 0x7fefa813ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T13:56:53.975 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.965+0000 7fefae2d6700 1 -- 172.21.15.40:0/1722994594 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa80ad8e0 con 0x7fefa8144390 2020-11-20T13:56:53.976 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.965+0000 7fefae2d6700 1 --2- 172.21.15.40:0/1722994594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefa8144390 0x7fefa80ad1d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7fefa80b71f0 tx=0x7fef98007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:53.976 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.966+0000 7fefa6ffd700 1 -- 172.21.15.40:0/1722994594 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef98013410 con 0x7fefa8144390 2020-11-20T13:56:53.976 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.966+0000 7fefa6ffd700 1 -- 172.21.15.40:0/1722994594 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef98013570 con 0x7fefa8144390 2020-11-20T13:56:53.976 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.966+0000 7fefafd39700 1 -- 172.21.15.40:0/1722994594 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefa80ae360 con 0x7fefa8144390 2020-11-20T13:56:53.976 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.966+0000 7fefa6ffd700 1 -- 172.21.15.40:0/1722994594 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef980136e0 con 0x7fefa8144390 2020-11-20T13:56:53.977 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.966+0000 7fefafd39700 1 -- 172.21.15.40:0/1722994594 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefa80ae830 con 0x7fefa8144390 2020-11-20T13:56:53.977 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.967+0000 7fefa6ffd700 1 -- 172.21.15.40:0/1722994594 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fef98010030 con 0x7fefa8144390 2020-11-20T13:56:53.977 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.967+0000 7fefafd39700 1 -- 172.21.15.40:0/1722994594 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefa80aeae0 con 0x7fefa8144390 2020-11-20T13:56:53.977 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.967+0000 7fefa6ffd700 1 --2- 172.21.15.40:0/1722994594 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fef88051320 0x7fef880537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:53.977 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.967+0000 7fefa6ffd700 1 -- 172.21.15.40:0/1722994594 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fef98068850 con 0x7fefa8144390 2020-11-20T13:56:53.978 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.967+0000 7fefadad5700 1 --2- 172.21.15.40:0/1722994594 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fef88051320 0x7fef880537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:53.978 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.967+0000 7fefadad5700 1 --2- 172.21.15.40:0/1722994594 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fef88051320 0x7fef880537e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7fefa001e000 tx=0x7fefa80b0780).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:53.978 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:53.971+0000 7fefa6ffd700 1 -- 172.21.15.40:0/1722994594 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fefa80aeae0 con 0x7fefa8144390 2020-11-20T13:56:54.134 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.132+0000 7fefafd39700 1 -- 172.21.15.40:0/1722994594 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fefa80aeae0 con 0x7fefa8144390 2020-11-20T13:56:54.134 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.132+0000 7fefa6ffd700 1 -- 172.21.15.40:0/1722994594 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v24) v1 ==== 110+0+10 (secure 0 0 0) 0x7fefa80aeae0 con 0x7fefa8144390 2020-11-20T13:56:54.134 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2020-11-20T13:56:54.135 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.134+0000 7fefafd39700 1 -- 172.21.15.40:0/1722994594 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fef88051320 msgr2=0x7fef880537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:54.135 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.134+0000 7fefafd39700 1 --2- 172.21.15.40:0/1722994594 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fef88051320 0x7fef880537e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7fefa001e000 tx=0x7fefa80b0780).stop 2020-11-20T13:56:54.136 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.134+0000 7fefafd39700 1 -- 172.21.15.40:0/1722994594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefa8144390 msgr2=0x7fefa80ad1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:54.136 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.134+0000 7fefafd39700 1 --2- 172.21.15.40:0/1722994594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefa8144390 0x7fefa80ad1d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7fefa80b71f0 tx=0x7fef98007ad0).stop 2020-11-20T13:56:54.136 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.134+0000 7fefafd39700 1 -- 172.21.15.40:0/1722994594 shutdown_connections 2020-11-20T13:56:54.142 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.134+0000 7fefafd39700 1 --2- 172.21.15.40:0/1722994594 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fef88051320 0x7fef880537e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.143 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.134+0000 7fefafd39700 1 --2- 172.21.15.40:0/1722994594 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fefa80b3960 0x7fefa8152f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.143 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.134+0000 7fefafd39700 1 --2- 172.21.15.40:0/1722994594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefa8144390 0x7fefa80ad1d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.143 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.134+0000 7fefafd39700 1 --2- 172.21.15.40:0/1722994594 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fefa80b5bf0 0x7fefa813ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T13:56:54.143 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.134+0000 7fefafd39700 1 -- 172.21.15.40:0/1722994594 >> 172.21.15.40:0/1722994594 conn(0x7fefa80ba1d0 msgr2=0x7fefa80b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:54.143 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.134+0000 7fefafd39700 1 -- 172.21.15.40:0/1722994594 shutdown_connections 2020-11-20T13:56:54.144 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.134+0000 7fefafd39700 1 -- 172.21.15.40:0/1722994594 wait complete. 2020-11-20T13:56:54.147 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-20T13:56:54.264 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.263+0000 7f0b795d4700 1 Processor -- start 2020-11-20T13:56:54.265 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.263+0000 7f0b795d4700 1 -- start start 2020-11-20T13:56:54.276 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.264+0000 7f0b795d4700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0b7405c510 0x7f0b7414b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:54.277 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.264+0000 7f0b795d4700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0b74061f90 0x7f0b7412d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:54.277 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.264+0000 7f0b795d4700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b7414b990 con 0x7f0b7412d6d0 2020-11-20T13:56:54.277 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.264+0000 7f0b795d4700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b74061310 con 0x7f0b7405c510 2020-11-20T13:56:54.277 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.264+0000 7f0b795d4700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b740615b0 con 0x7f0b74061f90 2020-11-20T13:56:54.278 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.264+0000 7f0b72ffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0b7405c510 0x7f0b7414b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:54.278 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.264+0000 7f0b72ffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0b7405c510 0x7f0b7414b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55602/0 (socket says 172.21.15.40:55602) 2020-11-20T13:56:54.278 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.264+0000 7f0b72ffd700 1 -- 172.21.15.40:0/45301482 learned_addr learned my addr 172.21.15.40:0/45301482 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:54.278 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.265+0000 7f0b727fc700 1 --2- 172.21.15.40:0/45301482 >> v2:172.21.15.116:3300/0 conn(0x7f0b74061f90 0x7f0b7412d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:54.278 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.265+0000 7f0b72ffd700 1 -- 172.21.15.40:0/45301482 >> v2:172.21.15.116:3300/0 conn(0x7f0b74061f90 msgr2=0x7f0b7412d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:54.279 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.265+0000 7f0b72ffd700 1 --2- 172.21.15.40:0/45301482 >> v2:172.21.15.116:3300/0 conn(0x7f0b74061f90 0x7f0b7412d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.279 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.265+0000 7f0b72ffd700 1 -- 172.21.15.40:0/45301482 >> v1:172.21.15.116:6789/0 conn(0x7f0b7412d6d0 legacy=0x7f0b74147520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:54.279 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.265+0000 7f0b72ffd700 1 -- 172.21.15.40:0/45301482 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b74061850 con 0x7f0b7405c510 2020-11-20T13:56:54.279 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.265+0000 7f0b727fc700 1 --2- 172.21.15.40:0/45301482 >> v2:172.21.15.116:3300/0 conn(0x7f0b74061f90 0x7f0b7412d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:54.279 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.265+0000 7f0b72ffd700 1 --2- 172.21.15.40:0/45301482 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b7405c510 0x7f0b7414b4a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f0b68002f30 tx=0x7f0b68006e80).ready entity=mon.0 client_cookie=410ff20ef77aad06 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:54.280 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.265+0000 7f0b71ffb700 1 -- 172.21.15.40:0/45301482 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b680097f0 con 0x7f0b7405c510 2020-11-20T13:56:54.280 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.265+0000 7f0b71ffb700 1 -- 172.21.15.40:0/45301482 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b68009950 con 0x7f0b7405c510 2020-11-20T13:56:54.285 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.265+0000 7f0b71ffb700 1 -- 172.21.15.40:0/45301482 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b68009cb0 con 0x7f0b7405c510 2020-11-20T13:56:54.286 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.265+0000 7f0b795d4700 1 -- 172.21.15.40:0/45301482 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b7405c510 msgr2=0x7f0b7414b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:54.286 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.265+0000 7f0b795d4700 1 --2- 172.21.15.40:0/45301482 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b7405c510 0x7f0b7414b4a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f0b68002f30 tx=0x7f0b68006e80).stop 2020-11-20T13:56:54.286 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.266+0000 7f0b795d4700 1 -- 172.21.15.40:0/45301482 shutdown_connections 2020-11-20T13:56:54.286 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.266+0000 7f0b795d4700 1 --2- 172.21.15.40:0/45301482 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b7405c510 0x7f0b7414b4a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.286 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.266+0000 7f0b795d4700 1 --2- 172.21.15.40:0/45301482 >> v2:172.21.15.116:3300/0 conn(0x7f0b74061f90 0x7f0b7412d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.287 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.266+0000 7f0b795d4700 1 -- 172.21.15.40:0/45301482 >> 172.21.15.40:0/45301482 conn(0x7f0b740bdb70 msgr2=0x7f0b740bdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:54.287 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.266+0000 7f0b795d4700 1 -- 172.21.15.40:0/45301482 shutdown_connections 2020-11-20T13:56:54.287 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.266+0000 7f0b795d4700 1 -- 172.21.15.40:0/45301482 wait complete. 2020-11-20T13:56:54.287 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.267+0000 7f0b795d4700 1 Processor -- start 2020-11-20T13:56:54.287 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.267+0000 7f0b795d4700 1 -- start start 2020-11-20T13:56:54.288 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.267+0000 7f0b795d4700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b7405c510 0x7f0b7415a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:54.288 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.267+0000 7f0b795d4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b74061f90 0x7f0b740ad6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:54.288 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.267+0000 7f0b795d4700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0b7412d6d0 0x7f0b74149b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:54.288 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.267+0000 7f0b795d4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7415ab60 con 0x7f0b74061f90 2020-11-20T13:56:54.288 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.267+0000 7f0b795d4700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b740adbb0 con 0x7f0b7405c510 2020-11-20T13:56:54.289 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.267+0000 7f0b795d4700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b74159f30 con 0x7f0b7412d6d0 2020-11-20T13:56:54.289 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.267+0000 7f0b727fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b74061f90 0x7f0b740ad6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:54.289 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.267+0000 7f0b727fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b74061f90 0x7f0b740ad6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55608/0 (socket says 172.21.15.40:55608) 2020-11-20T13:56:54.289 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.267+0000 7f0b727fc700 1 -- 172.21.15.40:0/2891520755 learned_addr learned my addr 172.21.15.40:0/2891520755 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:54.291 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.268+0000 7f0b72ffd700 1 --2- 172.21.15.40:0/2891520755 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b7405c510 0x7f0b7415a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:54.291 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.268+0000 7f0b737fe700 1 --2- 172.21.15.40:0/2891520755 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0b7412d6d0 0x7f0b74149b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:54.291 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.268+0000 7f0b727fc700 1 -- 172.21.15.40:0/2891520755 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0b7412d6d0 msgr2=0x7f0b74149b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:54.292 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.268+0000 7f0b727fc700 1 --2- 172.21.15.40:0/2891520755 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0b7412d6d0 0x7f0b74149b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.292 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.268+0000 7f0b727fc700 1 -- 172.21.15.40:0/2891520755 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b7405c510 msgr2=0x7f0b7415a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:54.292 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.268+0000 7f0b727fc700 1 --2- 172.21.15.40:0/2891520755 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b7405c510 0x7f0b7415a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.292 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.268+0000 7f0b727fc700 1 -- 172.21.15.40:0/2891520755 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b7415a1d0 con 0x7f0b74061f90 2020-11-20T13:56:54.292 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.268+0000 7f0b737fe700 1 --2- 172.21.15.40:0/2891520755 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0b7412d6d0 0x7f0b74149b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:54.293 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.268+0000 7f0b727fc700 1 --2- 172.21.15.40:0/2891520755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b74061f90 0x7f0b740ad6c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f0b64007000 tx=0x7f0b64008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:54.293 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.268+0000 7f0b63fff700 1 -- 172.21.15.40:0/2891520755 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b64010580 con 0x7f0b74061f90 2020-11-20T13:56:54.293 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.268+0000 7f0b63fff700 1 -- 172.21.15.40:0/2891520755 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b640106e0 con 0x7f0b74061f90 2020-11-20T13:56:54.293 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.268+0000 7f0b795d4700 1 -- 172.21.15.40:0/2891520755 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b7414a010 con 0x7f0b74061f90 2020-11-20T13:56:54.294 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.268+0000 7f0b63fff700 1 -- 172.21.15.40:0/2891520755 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b64010a40 con 0x7f0b74061f90 2020-11-20T13:56:54.294 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.268+0000 7f0b795d4700 1 -- 172.21.15.40:0/2891520755 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b740b02b0 con 0x7f0b74061f90 2020-11-20T13:56:54.294 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.269+0000 7f0b795d4700 1 -- 172.21.15.40:0/2891520755 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b7415a360 con 0x7f0b74061f90 2020-11-20T13:56:54.294 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.269+0000 7f0b63fff700 1 -- 172.21.15.40:0/2891520755 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f0b64010ba0 con 0x7f0b74061f90 2020-11-20T13:56:54.294 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.269+0000 7f0b63fff700 1 --2- 172.21.15.40:0/2891520755 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f0b50051320 0x7f0b500537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:54.295 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.269+0000 7f0b63fff700 1 -- 172.21.15.40:0/2891520755 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f0b64069240 con 0x7f0b74061f90 2020-11-20T13:56:54.295 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.270+0000 7f0b72ffd700 1 --2- 172.21.15.40:0/2891520755 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f0b50051320 0x7f0b500537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:54.295 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.270+0000 7f0b72ffd700 1 --2- 172.21.15.40:0/2891520755 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f0b50051320 0x7f0b500537e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f0b6801c000 tx=0x7f0b68015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:54.295 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.273+0000 7f0b63fff700 1 -- 172.21.15.40:0/2891520755 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f0b64041400 con 0x7f0b74061f90 2020-11-20T13:56:54.435 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.434+0000 7f0b795d4700 1 -- 172.21.15.40:0/2891520755 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0b74057050 con 0x7f0b74061f90 2020-11-20T13:56:54.436 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.435+0000 7f0b63fff700 1 -- 172.21.15.40:0/2891520755 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0b64044a20 con 0x7f0b74061f90 2020-11-20T13:56:54.436 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2020-11-20T13:56:54.437 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.436+0000 7f0b795d4700 1 -- 172.21.15.40:0/2891520755 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f0b50051320 msgr2=0x7f0b500537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:54.437 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.436+0000 7f0b795d4700 1 --2- 172.21.15.40:0/2891520755 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f0b50051320 0x7f0b500537e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f0b6801c000 tx=0x7f0b68015000).stop 2020-11-20T13:56:54.437 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.436+0000 7f0b795d4700 1 -- 172.21.15.40:0/2891520755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b74061f90 msgr2=0x7f0b740ad6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:54.438 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.436+0000 7f0b795d4700 1 --2- 172.21.15.40:0/2891520755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b74061f90 0x7f0b740ad6c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f0b64007000 tx=0x7f0b64008040).stop 2020-11-20T13:56:54.438 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.436+0000 7f0b795d4700 1 -- 172.21.15.40:0/2891520755 shutdown_connections 2020-11-20T13:56:54.438 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.436+0000 7f0b795d4700 1 --2- 172.21.15.40:0/2891520755 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f0b50051320 0x7f0b500537e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.438 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.436+0000 7f0b795d4700 1 --2- 172.21.15.40:0/2891520755 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0b7412d6d0 0x7f0b74149b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.439 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.436+0000 7f0b795d4700 1 --2- 172.21.15.40:0/2891520755 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b7405c510 0x7f0b7415a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.439 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.436+0000 7f0b795d4700 1 --2- 172.21.15.40:0/2891520755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b74061f90 0x7f0b740ad6c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.439 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.436+0000 7f0b795d4700 1 -- 172.21.15.40:0/2891520755 >> 172.21.15.40:0/2891520755 conn(0x7f0b740bdb70 msgr2=0x7f0b740b8510 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:54.439 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.436+0000 7f0b795d4700 1 -- 172.21.15.40:0/2891520755 shutdown_connections 2020-11-20T13:56:54.440 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.436+0000 7f0b795d4700 1 -- 172.21.15.40:0/2891520755 wait complete. 2020-11-20T13:56:54.447 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-20T13:56:54.567 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.564+0000 7f3b9a6d5700 1 Processor -- start 2020-11-20T13:56:54.567 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.564+0000 7f3b9a6d5700 1 -- start start 2020-11-20T13:56:54.568 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.564+0000 7f3b9a6d5700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3b9413f400 0x7f3b940b3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:54.568 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.564+0000 7f3b9a6d5700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b941489e0 con 0x7f3b940b6900 2020-11-20T13:56:54.568 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.564+0000 7f3b9a6d5700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b94148c80 con 0x7f3b9405fa50 2020-11-20T13:56:54.568 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.564+0000 7f3b9a6d5700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9405f5c0 con 0x7f3b9413f400 2020-11-20T13:56:54.568 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.564+0000 7f3b98c72700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3b940b6900 0x7f3b94145290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46210/0 (socket says 172.21.15.40:46210) 2020-11-20T13:56:54.569 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.564+0000 7f3b98c72700 1 -- 172.21.15.40:0/2931780811 learned_addr learned my addr 172.21.15.40:0/2931780811 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:54.569 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.565+0000 7f3b93fff700 1 --2- 172.21.15.40:0/2931780811 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3b9413f400 0x7f3b940b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:54.569 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.565+0000 7f3b937fe700 1 -- 172.21.15.40:0/2931780811 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3524245216 0 0) 0x7f3b941489e0 con 0x7f3b940b6900 2020-11-20T13:56:54.569 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.565+0000 7f3b937fe700 1 -- 172.21.15.40:0/2931780811 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b74003230 con 0x7f3b940b6900 2020-11-20T13:56:54.570 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.565+0000 7f3b937fe700 1 -- 172.21.15.40:0/2931780811 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 453230983 0 0) 0x7f3b94148c80 con 0x7f3b9405fa50 2020-11-20T13:56:54.570 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.565+0000 7f3b937fe700 1 -- 172.21.15.40:0/2931780811 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b941489e0 con 0x7f3b9405fa50 2020-11-20T13:56:54.570 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.565+0000 7f3b937fe700 1 -- 172.21.15.40:0/2931780811 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2349533645 0 0) 0x7f3b74003230 con 0x7f3b940b6900 2020-11-20T13:56:54.570 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.565+0000 7f3b93fff700 1 -- 172.21.15.40:0/2931780811 >> v1:172.21.15.116:6789/0 conn(0x7f3b9405fa50 legacy=0x7f3b94149120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:54.570 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.565+0000 7f3b93fff700 1 -- 172.21.15.40:0/2931780811 >> v1:172.21.15.40:6789/0 conn(0x7f3b940b6900 legacy=0x7f3b94145290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:54.571 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.565+0000 7f3b93fff700 1 -- 172.21.15.40:0/2931780811 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b940b6d90 con 0x7f3b9413f400 2020-11-20T13:56:54.571 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.565+0000 7f3b93fff700 1 --2- 172.21.15.40:0/2931780811 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3b9413f400 0x7f3b940b3730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f3b8c001450 tx=0x7f3b84006d60).ready entity=mon.2 client_cookie=3d31f7ab422f1c25 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:54.571 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.565+0000 7f3b937fe700 1 -- 172.21.15.40:0/2931780811 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b8c001a50 con 0x7f3b9413f400 2020-11-20T13:56:54.572 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.565+0000 7f3b937fe700 1 -- 172.21.15.40:0/2931780811 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b8400f510 con 0x7f3b9413f400 2020-11-20T13:56:54.572 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.565+0000 7f3b937fe700 1 -- 172.21.15.40:0/2931780811 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b8400f680 con 0x7f3b9413f400 2020-11-20T13:56:54.572 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.565+0000 7f3b9a6d5700 1 -- 172.21.15.40:0/2931780811 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3b9413f400 msgr2=0x7f3b940b3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:54.572 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.565+0000 7f3b9a6d5700 1 --2- 172.21.15.40:0/2931780811 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3b9413f400 0x7f3b940b3730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f3b8c001450 tx=0x7f3b84006d60).stop 2020-11-20T13:56:54.572 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.566+0000 7f3b9a6d5700 1 -- 172.21.15.40:0/2931780811 shutdown_connections 2020-11-20T13:56:54.573 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.566+0000 7f3b9a6d5700 1 --2- 172.21.15.40:0/2931780811 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3b9413f400 0x7f3b940b3730 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.573 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.566+0000 7f3b9a6d5700 1 -- 172.21.15.40:0/2931780811 >> 172.21.15.40:0/2931780811 conn(0x7f3b940ba1d0 msgr2=0x7f3b940b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:54.573 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.566+0000 7f3b9a6d5700 1 -- 172.21.15.40:0/2931780811 shutdown_connections 2020-11-20T13:56:54.573 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.566+0000 7f3b9a6d5700 1 -- 172.21.15.40:0/2931780811 wait complete. 2020-11-20T13:56:54.576 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.567+0000 7f3b9a6d5700 1 Processor -- start 2020-11-20T13:56:54.576 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.567+0000 7f3b9a6d5700 1 -- start start 2020-11-20T13:56:54.576 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.568+0000 7f3b9a6d5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b9405fa50 0x7f3b94141ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:54.577 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.568+0000 7f3b9a6d5700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3b940b6900 0x7f3b94156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:54.577 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.568+0000 7f3b9a6d5700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b9413f400 0x7f3b9415a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:54.577 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.568+0000 7f3b9a6d5700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9415a8b0 con 0x7f3b9405fa50 2020-11-20T13:56:54.577 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.568+0000 7f3b9a6d5700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b941565b0 con 0x7f3b9413f400 2020-11-20T13:56:54.577 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.568+0000 7f3b9a6d5700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b94156850 con 0x7f3b940b6900 2020-11-20T13:56:54.578 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.568+0000 7f3b93fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b9405fa50 0x7f3b94141ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:54.578 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.568+0000 7f3b93fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b9405fa50 0x7f3b94141ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55618/0 (socket says 172.21.15.40:55618) 2020-11-20T13:56:54.578 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.568+0000 7f3b93fff700 1 -- 172.21.15.40:0/2220615891 learned_addr learned my addr 172.21.15.40:0/2220615891 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:54.578 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.568+0000 7f3b93fff700 1 -- 172.21.15.40:0/2220615891 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3b940b6900 msgr2=0x7f3b94156f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:54.578 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.568+0000 7f3b93fff700 1 --2- 172.21.15.40:0/2220615891 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3b940b6900 0x7f3b94156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T13:56:54.579 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.568+0000 7f3b93fff700 1 -- 172.21.15.40:0/2220615891 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b9413f400 msgr2=0x7f3b9415a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:54.579 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.568+0000 7f3b93fff700 1 --2- 172.21.15.40:0/2220615891 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b9413f400 0x7f3b9415a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T13:56:54.579 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.568+0000 7f3b93fff700 1 -- 172.21.15.40:0/2220615891 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b94156af0 con 0x7f3b9405fa50 2020-11-20T13:56:54.579 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.568+0000 7f3b93fff700 1 --2- 172.21.15.40:0/2220615891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b9405fa50 0x7f3b94141ec0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f3b8c001450 tx=0x7f3b84016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:54.579 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.568+0000 7f3b91ffb700 1 -- 172.21.15.40:0/2220615891 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b8400fc90 con 0x7f3b9405fa50 2020-11-20T13:56:54.580 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.568+0000 7f3b91ffb700 1 -- 172.21.15.40:0/2220615891 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b8400fdf0 con 0x7f3b9405fa50 2020-11-20T13:56:54.580 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.568+0000 7f3b91ffb700 1 -- 172.21.15.40:0/2220615891 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b840058b0 con 0x7f3b9405fa50 2020-11-20T13:56:54.580 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.568+0000 7f3b9a6d5700 1 -- 172.21.15.40:0/2220615891 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b94143d40 con 0x7f3b9405fa50 2020-11-20T13:56:54.580 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.568+0000 7f3b9a6d5700 1 -- 172.21.15.40:0/2220615891 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b94147e20 con 0x7f3b9405fa50 2020-11-20T13:56:54.580 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.569+0000 7f3b9a6d5700 1 -- 172.21.15.40:0/2220615891 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b94057050 con 0x7f3b9405fa50 2020-11-20T13:56:54.581 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.569+0000 7f3b91ffb700 1 -- 172.21.15.40:0/2220615891 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f3b84026030 con 0x7f3b9405fa50 2020-11-20T13:56:54.581 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.570+0000 7f3b91ffb700 1 --2- 172.21.15.40:0/2220615891 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f3b780513c0 0x7f3b78053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:54.581 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.570+0000 7f3b91ffb700 1 -- 172.21.15.40:0/2220615891 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f3b84069510 con 0x7f3b9405fa50 2020-11-20T13:56:54.582 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.570+0000 7f3b8bfff700 1 --2- 172.21.15.40:0/2220615891 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f3b780513c0 0x7f3b78053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:54.582 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.570+0000 7f3b8bfff700 1 --2- 172.21.15.40:0/2220615891 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f3b780513c0 0x7f3b78053880 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f3b7c0010d0 tx=0x7f3b7c007650).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:54.583 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.573+0000 7f3b91ffb700 1 -- 172.21.15.40:0/2220615891 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f3b84041730 con 0x7f3b9405fa50 2020-11-20T13:56:54.735 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.733+0000 7f3b9a6d5700 1 -- 172.21.15.40:0/2220615891 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3b940b5fd0 con 0x7f3b9405fa50 2020-11-20T13:56:54.735 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.734+0000 7f3b91ffb700 1 -- 172.21.15.40:0/2220615891 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+10 (secure 0 0 0) 0x7f3b840150a0 con 0x7f3b9405fa50 2020-11-20T13:56:54.735 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2020-11-20T13:56:54.736 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.735+0000 7f3b9a6d5700 1 -- 172.21.15.40:0/2220615891 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f3b780513c0 msgr2=0x7f3b78053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:54.737 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.735+0000 7f3b9a6d5700 1 --2- 172.21.15.40:0/2220615891 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f3b780513c0 0x7f3b78053880 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f3b7c0010d0 tx=0x7f3b7c007650).stop 2020-11-20T13:56:54.737 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.735+0000 7f3b9a6d5700 1 -- 172.21.15.40:0/2220615891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b9405fa50 msgr2=0x7f3b94141ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:54.737 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.735+0000 7f3b9a6d5700 1 --2- 172.21.15.40:0/2220615891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b9405fa50 0x7f3b94141ec0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f3b8c001450 tx=0x7f3b84016000).stop 2020-11-20T13:56:54.737 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.735+0000 7f3b9a6d5700 1 -- 172.21.15.40:0/2220615891 shutdown_connections 2020-11-20T13:56:54.738 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.735+0000 7f3b9a6d5700 1 --2- 172.21.15.40:0/2220615891 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f3b780513c0 0x7f3b78053880 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.739 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.735+0000 7f3b9a6d5700 1 --2- 172.21.15.40:0/2220615891 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b9413f400 0x7f3b9415a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T13:56:54.739 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.735+0000 7f3b9a6d5700 1 --2- 172.21.15.40:0/2220615891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b9405fa50 0x7f3b94141ec0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.740 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.735+0000 7f3b9a6d5700 1 --2- 172.21.15.40:0/2220615891 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3b940b6900 0x7f3b94156f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T13:56:54.740 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.735+0000 7f3b9a6d5700 1 -- 172.21.15.40:0/2220615891 >> 172.21.15.40:0/2220615891 conn(0x7f3b940ba1d0 msgr2=0x7f3b940ae310 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:54.740 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.735+0000 7f3b9a6d5700 1 -- 172.21.15.40:0/2220615891 shutdown_connections 2020-11-20T13:56:54.741 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.735+0000 7f3b9a6d5700 1 -- 172.21.15.40:0/2220615891 wait complete. 2020-11-20T13:56:54.747 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-20T13:56:54.865 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.863+0000 7f64ea616700 1 Processor -- start 2020-11-20T13:56:54.865 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.863+0000 7f64ea616700 1 -- start start 2020-11-20T13:56:54.868 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.864+0000 7f64ea616700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f64e40b6c80 0x7f64e40b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:54.868 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.864+0000 7f64ea616700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64e40b0d10 con 0x7f64e4144bd0 2020-11-20T13:56:54.871 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.864+0000 7f64ea616700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64e40b0fb0 con 0x7f64e40b1450 2020-11-20T13:56:54.872 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.864+0000 7f64ea616700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64e4145060 con 0x7f64e40b6c80 2020-11-20T13:56:54.872 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.864+0000 7f64e37fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f64e4144bd0 0x7f64e40b40f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46222/0 (socket says 172.21.15.40:46222) 2020-11-20T13:56:54.872 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.864+0000 7f64e37fe700 1 -- 172.21.15.40:0/1527729975 learned_addr learned my addr 172.21.15.40:0/1527729975 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:54.873 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.865+0000 7f64e3fff700 1 --2- 172.21.15.40:0/1527729975 >> v2:172.21.15.116:3300/0 conn(0x7f64e40b6c80 0x7f64e40b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:54.873 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.865+0000 7f64e2ffd700 1 -- 172.21.15.40:0/1527729975 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4076129513 0 0) 0x7f64e40b0d10 con 0x7f64e4144bd0 2020-11-20T13:56:54.873 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.865+0000 7f64e2ffd700 1 -- 172.21.15.40:0/1527729975 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64d0003230 con 0x7f64e4144bd0 2020-11-20T13:56:54.873 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.865+0000 7f64e2ffd700 1 -- 172.21.15.40:0/1527729975 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3595827368 0 0) 0x7f64e40b0fb0 con 0x7f64e40b1450 2020-11-20T13:56:54.874 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.865+0000 7f64e2ffd700 1 -- 172.21.15.40:0/1527729975 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64e40b0d10 con 0x7f64e40b1450 2020-11-20T13:56:54.874 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.865+0000 7f64e2ffd700 1 -- 172.21.15.40:0/1527729975 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1372959813 0 0) 0x7f64d0003230 con 0x7f64e4144bd0 2020-11-20T13:56:54.874 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.865+0000 7f64e2ffd700 1 -- 172.21.15.40:0/1527729975 >> v2:172.21.15.116:3300/0 conn(0x7f64e40b6c80 msgr2=0x7f64e40b31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:54.874 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.865+0000 7f64e2ffd700 1 --2- 172.21.15.40:0/1527729975 >> v2:172.21.15.116:3300/0 conn(0x7f64e40b6c80 0x7f64e40b31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.875 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.865+0000 7f64e2ffd700 1 -- 172.21.15.40:0/1527729975 >> v1:172.21.15.116:6789/0 conn(0x7f64e40b1450 legacy=0x7f64e412ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:54.875 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.865+0000 7f64e2ffd700 1 -- 172.21.15.40:0/1527729975 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e40b7060 con 0x7f64e4144bd0 2020-11-20T13:56:54.875 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.865+0000 7f64e2ffd700 1 -- 172.21.15.40:0/1527729975 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2366654341 0 0) 0x7f64d80016d0 con 0x7f64e4144bd0 2020-11-20T13:56:54.875 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.865+0000 7f64e2ffd700 1 -- 172.21.15.40:0/1527729975 >> v1:172.21.15.40:6789/0 conn(0x7f64e4144bd0 legacy=0x7f64e40b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:54.875 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.865+0000 7f64e2ffd700 1 --2- 172.21.15.40:0/1527729975 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64d0003900 0x7f64d0003d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:54.876 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.865+0000 7f64e2ffd700 1 --2- 172.21.15.40:0/1527729975 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64d00044d0 0x7f64d0004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:54.876 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.865+0000 7f64e2ffd700 1 --2- 172.21.15.40:0/1527729975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64d0004e40 0x7f64d000b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:54.876 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.865+0000 7f64e2ffd700 1 -- 172.21.15.40:0/1527729975 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e4145060 con 0x7f64d0004e40 2020-11-20T13:56:54.876 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.865+0000 7f64e2ffd700 1 -- 172.21.15.40:0/1527729975 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d000bc40 con 0x7f64d00044d0 2020-11-20T13:56:54.877 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.865+0000 7f64e2ffd700 1 -- 172.21.15.40:0/1527729975 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d000e010 con 0x7f64d0003900 2020-11-20T13:56:54.877 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.865+0000 7f64e8bb3700 1 --2- 172.21.15.40:0/1527729975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64d0004e40 0x7f64d000b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:54.877 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.865+0000 7f64e37fe700 1 --2- 172.21.15.40:0/1527729975 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64d00044d0 0x7f64d0004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:54.877 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.865+0000 7f64e3fff700 1 --2- 172.21.15.40:0/1527729975 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64d0003900 0x7f64d0003d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:54.877 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.865+0000 7f64e8bb3700 1 -- 172.21.15.40:0/1527729975 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64d0003900 msgr2=0x7f64d0003d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:54.878 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.865+0000 7f64e8bb3700 1 --2- 172.21.15.40:0/1527729975 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64d0003900 0x7f64d0003d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.878 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.865+0000 7f64e8bb3700 1 -- 172.21.15.40:0/1527729975 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64d00044d0 msgr2=0x7f64d0004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:54.878 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.865+0000 7f64e8bb3700 1 --2- 172.21.15.40:0/1527729975 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64d00044d0 0x7f64d0004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.878 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.865+0000 7f64e8bb3700 1 -- 172.21.15.40:0/1527729975 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d0003230 con 0x7f64d0004e40 2020-11-20T13:56:54.878 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.865+0000 7f64e8bb3700 1 -- 172.21.15.40:0/1527729975 reap_dead start 2020-11-20T13:56:54.879 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.866+0000 7f64e8bb3700 1 --2- 172.21.15.40:0/1527729975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64d0004e40 0x7f64d000b720 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f64dc001450 tx=0x7f64dc007e50).ready entity=mon.0 client_cookie=7b4bd269f098d75b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:54.879 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.866+0000 7f64e2ffd700 1 -- 172.21.15.40:0/1527729975 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f64dc00cb50 con 0x7f64d0004e40 2020-11-20T13:56:54.879 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.866+0000 7f64e2ffd700 1 -- 172.21.15.40:0/1527729975 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64dc00ccb0 con 0x7f64d0004e40 2020-11-20T13:56:54.879 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.866+0000 7f64e2ffd700 1 -- 172.21.15.40:0/1527729975 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f64dc0125e0 con 0x7f64d0004e40 2020-11-20T13:56:54.880 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.866+0000 7f64ea616700 1 -- 172.21.15.40:0/1527729975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64d0004e40 msgr2=0x7f64d000b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:54.880 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.866+0000 7f64ea616700 1 --2- 172.21.15.40:0/1527729975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64d0004e40 0x7f64d000b720 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f64dc001450 tx=0x7f64dc007e50).stop 2020-11-20T13:56:54.880 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.866+0000 7f64ea616700 1 -- 172.21.15.40:0/1527729975 shutdown_connections 2020-11-20T13:56:54.880 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.866+0000 7f64ea616700 1 --2- 172.21.15.40:0/1527729975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64d0004e40 0x7f64d000b720 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.880 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.866+0000 7f64ea616700 1 -- 172.21.15.40:0/1527729975 >> 172.21.15.40:0/1527729975 conn(0x7f64e40ba1d0 msgr2=0x7f64e40b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:54.881 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.866+0000 7f64ea616700 1 -- 172.21.15.40:0/1527729975 shutdown_connections 2020-11-20T13:56:54.881 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.866+0000 7f64ea616700 1 -- 172.21.15.40:0/1527729975 wait complete. 2020-11-20T13:56:54.881 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.867+0000 7f64ea616700 1 Processor -- start 2020-11-20T13:56:54.881 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.867+0000 7f64ea616700 1 -- start start 2020-11-20T13:56:54.883 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.868+0000 7f64ea616700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64d0004e40 0x7f64e4151c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:54.884 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.868+0000 7f64ea616700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64e41551c0 0x7f64e40aef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:54.884 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.868+0000 7f64ea616700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64e41433c0 0x7f64e4143870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:54.884 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.868+0000 7f64ea616700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e41560b0 con 0x7f64d0004e40 2020-11-20T13:56:54.884 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.868+0000 7f64ea616700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e41556d0 con 0x7f64e41551c0 2020-11-20T13:56:54.902 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.868+0000 7f64ea616700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64e4155970 con 0x7f64e41433c0 2020-11-20T13:56:54.902 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.868+0000 7f64e3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64d0004e40 0x7f64e4151c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:54.902 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.868+0000 7f64e3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64d0004e40 0x7f64e4151c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55638/0 (socket says 172.21.15.40:55638) 2020-11-20T13:56:54.903 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.868+0000 7f64e3fff700 1 -- 172.21.15.40:0/3075352799 learned_addr learned my addr 172.21.15.40:0/3075352799 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:54.903 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.868+0000 7f64e8bb3700 1 --2- 172.21.15.40:0/3075352799 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64e41433c0 0x7f64e4143870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:54.903 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.868+0000 7f64e37fe700 1 --2- 172.21.15.40:0/3075352799 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64e41551c0 0x7f64e40aef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:54.903 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.868+0000 7f64e3fff700 1 -- 172.21.15.40:0/3075352799 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64e41433c0 msgr2=0x7f64e4143870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:54.903 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.868+0000 7f64e3fff700 1 --2- 172.21.15.40:0/3075352799 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64e41433c0 0x7f64e4143870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.904 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.868+0000 7f64e3fff700 1 -- 172.21.15.40:0/3075352799 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64e41551c0 msgr2=0x7f64e40aef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:54.904 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.868+0000 7f64e3fff700 1 --2- 172.21.15.40:0/3075352799 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64e41551c0 0x7f64e40aef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.904 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.868+0000 7f64e3fff700 1 -- 172.21.15.40:0/3075352799 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e4155c10 con 0x7f64d0004e40 2020-11-20T13:56:54.904 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.869+0000 7f64e3fff700 1 --2- 172.21.15.40:0/3075352799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64d0004e40 0x7f64e4151c00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f64d4001e60 tx=0x7f64d4006d60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:54.904 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.869+0000 7f64e17fa700 1 -- 172.21.15.40:0/3075352799 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f64d4013680 con 0x7f64d0004e40 2020-11-20T13:56:54.905 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.869+0000 7f64e17fa700 1 -- 172.21.15.40:0/3075352799 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64d40137e0 con 0x7f64d0004e40 2020-11-20T13:56:54.905 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.869+0000 7f64ea616700 1 -- 172.21.15.40:0/3075352799 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64e412a470 con 0x7f64d0004e40 2020-11-20T13:56:54.905 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.869+0000 7f64e17fa700 1 -- 172.21.15.40:0/3075352799 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f64d4013950 con 0x7f64d0004e40 2020-11-20T13:56:54.905 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.869+0000 7f64ea616700 1 -- 172.21.15.40:0/3075352799 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e4128d60 con 0x7f64d0004e40 2020-11-20T13:56:54.905 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.870+0000 7f64ea616700 1 -- 172.21.15.40:0/3075352799 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64e4155da0 con 0x7f64d0004e40 2020-11-20T13:56:54.906 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.870+0000 7f64e17fa700 1 -- 172.21.15.40:0/3075352799 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f64d4024070 con 0x7f64d0004e40 2020-11-20T13:56:54.906 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.870+0000 7f64e17fa700 1 --2- 172.21.15.40:0/3075352799 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f64c40513c0 0x7f64c4053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:54.906 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.870+0000 7f64e17fa700 1 -- 172.21.15.40:0/3075352799 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f64d4069550 con 0x7f64d0004e40 2020-11-20T13:56:54.906 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.870+0000 7f64e37fe700 1 --2- 172.21.15.40:0/3075352799 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f64c40513c0 0x7f64c4053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:54.906 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.870+0000 7f64e37fe700 1 --2- 172.21.15.40:0/3075352799 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f64c40513c0 0x7f64c4053880 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f64d8005000 tx=0x7f64e40b0ea0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:54.907 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:54.873+0000 7f64e17fa700 1 -- 172.21.15.40:0/3075352799 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f64d4041770 con 0x7f64d0004e40 2020-11-20T13:56:55.038 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.033+0000 7f64ea616700 1 -- 172.21.15.40:0/3075352799 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f64e4057050 con 0x7f64d0004e40 2020-11-20T13:56:55.038 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.033+0000 7f64e17fa700 1 -- 172.21.15.40:0/3075352799 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+10 (secure 0 0 0) 0x7f64d4044d90 con 0x7f64d0004e40 2020-11-20T13:56:55.039 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.034+0000 7f64ea616700 1 -- 172.21.15.40:0/3075352799 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f64c40513c0 msgr2=0x7f64c4053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:55.039 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.034+0000 7f64ea616700 1 --2- 172.21.15.40:0/3075352799 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f64c40513c0 0x7f64c4053880 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f64d8005000 tx=0x7f64e40b0ea0).stop 2020-11-20T13:56:55.039 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.034+0000 7f64ea616700 1 -- 172.21.15.40:0/3075352799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64d0004e40 msgr2=0x7f64e4151c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:55.039 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.034+0000 7f64ea616700 1 --2- 172.21.15.40:0/3075352799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64d0004e40 0x7f64e4151c00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f64d4001e60 tx=0x7f64d4006d60).stop 2020-11-20T13:56:55.039 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.035+0000 7f64ea616700 1 -- 172.21.15.40:0/3075352799 shutdown_connections 2020-11-20T13:56:55.040 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.035+0000 7f64ea616700 1 --2- 172.21.15.40:0/3075352799 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f64c40513c0 0x7f64c4053880 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:55.040 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.035+0000 7f64ea616700 1 --2- 172.21.15.40:0/3075352799 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64e41433c0 0x7f64e4143870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:55.040 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.035+0000 7f64ea616700 1 --2- 172.21.15.40:0/3075352799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64d0004e40 0x7f64e4151c00 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:55.040 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.035+0000 7f64ea616700 1 --2- 172.21.15.40:0/3075352799 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64e41551c0 0x7f64e40aef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:55.042 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.035+0000 7f64ea616700 1 -- 172.21.15.40:0/3075352799 >> 172.21.15.40:0/3075352799 conn(0x7f64e40ba1d0 msgr2=0x7f64e40b2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:55.042 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.035+0000 7f64ea616700 1 -- 172.21.15.40:0/3075352799 shutdown_connections 2020-11-20T13:56:55.042 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.035+0000 7f64ea616700 1 -- 172.21.15.40:0/3075352799 wait complete. 2020-11-20T13:56:55.043 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2020-11-20T13:56:55.048 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-20T13:56:55.165 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.164+0000 7ff899432700 1 Processor -- start 2020-11-20T13:56:55.166 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.164+0000 7ff899432700 1 -- start start 2020-11-20T13:56:55.168 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.165+0000 7ff899432700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff8940b88c0 0x7ff8940b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:55.168 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.165+0000 7ff899432700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8940b6be0 con 0x7ff89409a280 2020-11-20T13:56:55.168 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.165+0000 7ff899432700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8940b4250 con 0x7ff89409b530 2020-11-20T13:56:55.169 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.165+0000 7ff899432700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff894099000 con 0x7ff8940b88c0 2020-11-20T13:56:55.169 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.165+0000 7ff8937fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff89409a280 0x7ff89414e390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46244/0 (socket says 172.21.15.40:46244) 2020-11-20T13:56:55.169 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.165+0000 7ff8937fe700 1 -- 172.21.15.40:0/3836006671 learned_addr learned my addr 172.21.15.40:0/3836006671 (peer_addr_for_me v1:172.21.15.40:0/0) 2020-11-20T13:56:55.169 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.165+0000 7ff892ffd700 1 --2- 172.21.15.40:0/3836006671 >> v2:172.21.15.116:3300/0 conn(0x7ff8940b88c0 0x7ff8940b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:55.170 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.165+0000 7ff891ffb700 1 -- 172.21.15.40:0/3836006671 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2112487586 0 0) 0x7ff8940b6be0 con 0x7ff89409a280 2020-11-20T13:56:55.170 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.165+0000 7ff891ffb700 1 -- 172.21.15.40:0/3836006671 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff878003230 con 0x7ff89409a280 2020-11-20T13:56:55.170 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.166+0000 7ff891ffb700 1 -- 172.21.15.40:0/3836006671 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1499667242 0 0) 0x7ff8940b4250 con 0x7ff89409b530 2020-11-20T13:56:55.170 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.166+0000 7ff891ffb700 1 -- 172.21.15.40:0/3836006671 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8940b6be0 con 0x7ff89409b530 2020-11-20T13:56:55.171 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.166+0000 7ff891ffb700 1 -- 172.21.15.40:0/3836006671 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1922208600 0 0) 0x7ff878003230 con 0x7ff89409a280 2020-11-20T13:56:55.171 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.166+0000 7ff892ffd700 1 -- 172.21.15.40:0/3836006671 >> v1:172.21.15.116:6789/0 conn(0x7ff89409b530 legacy=0x7ff89409b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:55.171 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.166+0000 7ff892ffd700 1 -- 172.21.15.40:0/3836006671 >> v1:172.21.15.40:6789/0 conn(0x7ff89409a280 legacy=0x7ff89414e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:55.171 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.166+0000 7ff892ffd700 1 -- 172.21.15.40:0/3836006671 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8940af250 con 0x7ff8940b88c0 2020-11-20T13:56:55.171 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.166+0000 7ff892ffd700 1 --2- 172.21.15.40:0/3836006671 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8940b88c0 0x7ff8940b8ca0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7ff88c001450 tx=0x7ff888006dc0).ready entity=mon.1 client_cookie=3969c20136d99720 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:55.172 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.166+0000 7ff891ffb700 1 -- 172.21.15.40:0/3836006671 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff88c001a50 con 0x7ff8940b88c0 2020-11-20T13:56:55.172 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.166+0000 7ff891ffb700 1 -- 172.21.15.40:0/3836006671 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff88800d610 con 0x7ff8940b88c0 2020-11-20T13:56:55.172 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.166+0000 7ff891ffb700 1 -- 172.21.15.40:0/3836006671 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff88800d780 con 0x7ff8940b88c0 2020-11-20T13:56:55.172 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.166+0000 7ff899432700 1 -- 172.21.15.40:0/3836006671 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8940b88c0 msgr2=0x7ff8940b8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:55.172 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.166+0000 7ff899432700 1 --2- 172.21.15.40:0/3836006671 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8940b88c0 0x7ff8940b8ca0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7ff88c001450 tx=0x7ff888006dc0).stop 2020-11-20T13:56:55.173 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.167+0000 7ff899432700 1 -- 172.21.15.40:0/3836006671 shutdown_connections 2020-11-20T13:56:55.173 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.167+0000 7ff899432700 1 --2- 172.21.15.40:0/3836006671 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8940b88c0 0x7ff8940b8ca0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:55.173 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.167+0000 7ff899432700 1 -- 172.21.15.40:0/3836006671 >> 172.21.15.40:0/3836006671 conn(0x7ff8940ba380 msgr2=0x7ff8940b6350 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:55.173 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.167+0000 7ff899432700 1 -- 172.21.15.40:0/3836006671 shutdown_connections 2020-11-20T13:56:55.174 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.167+0000 7ff899432700 1 -- 172.21.15.40:0/3836006671 wait complete. 2020-11-20T13:56:55.182 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.168+0000 7ff899432700 1 Processor -- start 2020-11-20T13:56:55.183 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.168+0000 7ff899432700 1 -- start start 2020-11-20T13:56:55.183 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.168+0000 7ff899432700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff89409a280 0x7ff8940665b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:55.183 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.168+0000 7ff899432700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff89409b530 0x7ff894067060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:55.183 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.168+0000 7ff899432700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff8940b88c0 0x7ff8940b33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:55.184 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.168+0000 7ff899432700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8940b5d90 con 0x7ff89409a280 2020-11-20T13:56:55.184 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.168+0000 7ff899432700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8940656e0 con 0x7ff89409b530 2020-11-20T13:56:55.184 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.168+0000 7ff899432700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff894065980 con 0x7ff8940b88c0 2020-11-20T13:56:55.184 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.168+0000 7ff892ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff89409a280 0x7ff8940665b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:55.185 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.168+0000 7ff892ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff89409a280 0x7ff8940665b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55650/0 (socket says 172.21.15.40:55650) 2020-11-20T13:56:55.185 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.168+0000 7ff892ffd700 1 -- 172.21.15.40:0/1108075700 learned_addr learned my addr 172.21.15.40:0/1108075700 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:55.185 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.169+0000 7ff8937fe700 1 --2- 172.21.15.40:0/1108075700 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff8940b88c0 0x7ff8940b33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:55.186 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.169+0000 7ff8927fc700 1 --2- 172.21.15.40:0/1108075700 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff89409b530 0x7ff894067060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:55.186 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.169+0000 7ff892ffd700 1 -- 172.21.15.40:0/1108075700 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff8940b88c0 msgr2=0x7ff8940b33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:55.186 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.169+0000 7ff892ffd700 1 --2- 172.21.15.40:0/1108075700 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff8940b88c0 0x7ff8940b33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:55.187 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.169+0000 7ff892ffd700 1 -- 172.21.15.40:0/1108075700 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff89409b530 msgr2=0x7ff894067060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:55.187 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.169+0000 7ff892ffd700 1 --2- 172.21.15.40:0/1108075700 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff89409b530 0x7ff894067060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:55.187 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.169+0000 7ff892ffd700 1 -- 172.21.15.40:0/1108075700 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff894065c20 con 0x7ff89409a280 2020-11-20T13:56:55.187 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.169+0000 7ff892ffd700 1 --2- 172.21.15.40:0/1108075700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff89409a280 0x7ff8940665b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7ff88c001450 tx=0x7ff888015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:55.187 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.169+0000 7ff883fff700 1 -- 172.21.15.40:0/1108075700 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff888011790 con 0x7ff89409a280 2020-11-20T13:56:55.188 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.169+0000 7ff883fff700 1 -- 172.21.15.40:0/1108075700 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8880118f0 con 0x7ff89409a280 2020-11-20T13:56:55.188 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.169+0000 7ff883fff700 1 -- 172.21.15.40:0/1108075700 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff888011c50 con 0x7ff89409a280 2020-11-20T13:56:55.188 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.169+0000 7ff899432700 1 -- 172.21.15.40:0/1108075700 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff894067550 con 0x7ff89409a280 2020-11-20T13:56:55.188 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.169+0000 7ff899432700 1 -- 172.21.15.40:0/1108075700 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff894065400 con 0x7ff89409a280 2020-11-20T13:56:55.188 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.170+0000 7ff899432700 1 -- 172.21.15.40:0/1108075700 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff894057050 con 0x7ff89409a280 2020-11-20T13:56:55.189 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.170+0000 7ff883fff700 1 -- 172.21.15.40:0/1108075700 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7ff888024030 con 0x7ff89409a280 2020-11-20T13:56:55.189 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.170+0000 7ff883fff700 1 --2- 172.21.15.40:0/1108075700 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7ff870051320 0x7ff8700537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:55.189 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.170+0000 7ff883fff700 1 -- 172.21.15.40:0/1108075700 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7ff888069ee0 con 0x7ff89409a280 2020-11-20T13:56:55.189 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.171+0000 7ff8927fc700 1 --2- 172.21.15.40:0/1108075700 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7ff870051320 0x7ff8700537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:55.190 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.171+0000 7ff8927fc700 1 --2- 172.21.15.40:0/1108075700 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7ff870051320 0x7ff8700537e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7ff88400d000 tx=0x7ff8840079f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:55.190 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.174+0000 7ff883fff700 1 -- 172.21.15.40:0/1108075700 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7ff888042140 con 0x7ff89409a280 2020-11-20T13:56:55.335 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.334+0000 7ff899432700 1 -- 172.21.15.40:0/1108075700 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff894065db0 con 0x7ff89409a280 2020-11-20T13:56:55.336 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.335+0000 7ff883fff700 1 -- 172.21.15.40:0/1108075700 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16347 (secure 0 0 0) 0x7ff888022020 con 0x7ff89409a280 2020-11-20T13:56:55.336 INFO:teuthology.orchestra.run.smithi040.stdout: 2020-11-20T13:56:55.336 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":24,"fsid":"da1ed737-5174-430f-a3c9-56f6bbfad13c","created":"2020-11-20T13:56:04.990651+0000","modified":"2020-11-20T13:56:25.428032+0000","last_up_change":"2020-11-20T13:56:13.893026+0000","last_in_change":"2020-11-20T13:56:13.893026+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T13:56:08.330017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T13:56:17.939684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T13:56:20.563740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T13:56:23.062623+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9ea4c667-8938-46d6-a5ca-092d55c1f79e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":22825},{"type":"v1","addr":"172.21.15.40:6818","nonce":22825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":22825},{"type":"v1","addr":"172.21.15.40:6822","nonce":22825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":22825},{"type":"v1","addr":"172.21.15.40:6830","nonce":22825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":22825},{"type":"v1","addr":"172.21.15.40:6826","nonce":22825}]},"public_addr":"172.21.15.40:6818/22825","cluster_addr":"172.21.15.40:6822/22825","heartbeat_back_addr":"172.21.15.40:6830/22825","heartbeat_front_addr":"172.21.15.40:6826/22825","state":["exists","up"]},{"osd":1,"uuid":"515708d9-5087-4e55-8d14-cdea1c00eacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":22823},{"type":"v1","addr":"172.21.15.40:6801","nonce":22823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":22823},{"type":"v1","addr":"172.21.15.40:6803","nonce":22823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":22823},{"type":"v1","addr":"172.21.15.40:6807","nonce":22823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":22823},{"type":"v1","addr":"172.21.15.40:6805","nonce":22823}]},"public_addr":"172.21.15.40:6801/22823","cluster_addr":"172.21.15.40:6803/22823","heartbeat_back_addr":"172.21.15.40:6807/22823","heartbeat_front_addr":"172.21.15.40:6805/22823","state":["exists","up"]},{"osd":2,"uuid":"4d86bca7-743e-4983-b35f-36676c405b36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":22824},{"type":"v1","addr":"172.21.15.40:6809","nonce":22824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":22824},{"type":"v1","addr":"172.21.15.40:6811","nonce":22824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":22824},{"type":"v1","addr":"172.21.15.40:6815","nonce":22824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":22824},{"type":"v1","addr":"172.21.15.40:6813","nonce":22824}]},"public_addr":"172.21.15.40:6809/22824","cluster_addr":"172.21.15.40:6811/22824","heartbeat_back_addr":"172.21.15.40:6815/22824","heartbeat_front_addr":"172.21.15.40:6813/22824","state":["exists","up"]},{"osd":3,"uuid":"f4dc726b-6b34-4e8b-8eeb-87060b58002d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":22826},{"type":"v1","addr":"172.21.15.40:6819","nonce":22826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":22826},{"type":"v1","addr":"172.21.15.40:6823","nonce":22826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":22826},{"type":"v1","addr":"172.21.15.40:6831","nonce":22826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":22826},{"type":"v1","addr":"172.21.15.40:6827","nonce":22826}]},"public_addr":"172.21.15.40:6819/22826","cluster_addr":"172.21.15.40:6823/22826","heartbeat_back_addr":"172.21.15.40:6831/22826","heartbeat_front_addr":"172.21.15.40:6827/22826","state":["exists","up"]},{"osd":4,"uuid":"25a6d53b-d1f0-4498-b7d5-ffa38661d403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":23181},{"type":"v1","addr":"172.21.15.116:6819","nonce":23181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":23181},{"type":"v1","addr":"172.21.15.116:6821","nonce":23181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":23181},{"type":"v1","addr":"172.21.15.116:6825","nonce":23181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":23181},{"type":"v1","addr":"172.21.15.116:6823","nonce":23181}]},"public_addr":"172.21.15.116:6819/23181","cluster_addr":"172.21.15.116:6821/23181","heartbeat_back_addr":"172.21.15.116:6825/23181","heartbeat_front_addr":"172.21.15.116:6823/23181","state":["exists","up"]},{"osd":5,"uuid":"7d7a5af8-229b-4351-9a21-a3d509cd4635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":23171},{"type":"v1","addr":"172.21.15.116:6803","nonce":23171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":23171},{"type":"v1","addr":"172.21.15.116:6805","nonce":23171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":23171},{"type":"v1","addr":"172.21.15.116:6809","nonce":23171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":23171},{"type":"v1","addr":"172.21.15.116:6807","nonce":23171}]},"public_addr":"172.21.15.116:6803/23171","cluster_addr":"172.21.15.116:6805/23171","heartbeat_back_addr":"172.21.15.116:6809/23171","heartbeat_front_addr":"172.21.15.116:6807/23171","state":["exists","up"]},{"osd":6,"uuid":"8247d248-a140-4e36-9796-9847a97df58d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":23172},{"type":"v1","addr":"172.21.15.116:6811","nonce":23172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":23172},{"type":"v1","addr":"172.21.15.116:6813","nonce":23172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":23172},{"type":"v1","addr":"172.21.15.116:6817","nonce":23172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":23172},{"type":"v1","addr":"172.21.15.116:6815","nonce":23172}]},"public_addr":"172.21.15.116:6811/23172","cluster_addr":"172.21.15.116:6813/23172","heartbeat_back_addr":"172.21.15.116:6817/23172","heartbeat_front_addr":"172.21.15.116:6815/23172","state":["exists","up"]},{"osd":7,"uuid":"472803c4-70b8-499d-b18a-71aaeec6aab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":23196},{"type":"v1","addr":"172.21.15.116:6827","nonce":23196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":23196},{"type":"v1","addr":"172.21.15.116:6829","nonce":23196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":23196},{"type":"v1","addr":"172.21.15.116:6833","nonce":23196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":23196},{"type":"v1","addr":"172.21.15.116:6831","nonce":23196}]},"public_addr":"172.21.15.116:6827/23196","cluster_addr":"172.21.15.116:6829/23196","heartbeat_back_addr":"172.21.15.116:6833/23196","heartbeat_front_addr":"172.21.15.116:6831/23196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.029357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:11.989797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.091097+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.014766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.070860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.071864+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.063400+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:56:12.109028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-20T13:56:55.339 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.336+0000 7ff899432700 1 -- 172.21.15.40:0/1108075700 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7ff870051320 msgr2=0x7ff8700537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:55.339 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.336+0000 7ff899432700 1 --2- 172.21.15.40:0/1108075700 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7ff870051320 0x7ff8700537e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7ff88400d000 tx=0x7ff8840079f0).stop 2020-11-20T13:56:55.339 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.336+0000 7ff899432700 1 -- 172.21.15.40:0/1108075700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff89409a280 msgr2=0x7ff8940665b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:55.339 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.336+0000 7ff899432700 1 --2- 172.21.15.40:0/1108075700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff89409a280 0x7ff8940665b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7ff88c001450 tx=0x7ff888015000).stop 2020-11-20T13:56:55.340 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.336+0000 7ff899432700 1 -- 172.21.15.40:0/1108075700 shutdown_connections 2020-11-20T13:56:55.340 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.336+0000 7ff899432700 1 --2- 172.21.15.40:0/1108075700 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7ff870051320 0x7ff8700537e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:55.340 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.336+0000 7ff899432700 1 --2- 172.21.15.40:0/1108075700 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff8940b88c0 0x7ff8940b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:55.340 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.336+0000 7ff899432700 1 --2- 172.21.15.40:0/1108075700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff89409a280 0x7ff8940665b0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:55.340 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.336+0000 7ff899432700 1 --2- 172.21.15.40:0/1108075700 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff89409b530 0x7ff894067060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:55.341 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.336+0000 7ff899432700 1 -- 172.21.15.40:0/1108075700 >> 172.21.15.40:0/1108075700 conn(0x7ff8940ba380 msgr2=0x7ff894142c40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:55.341 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.336+0000 7ff899432700 1 -- 172.21.15.40:0/1108075700 shutdown_connections 2020-11-20T13:56:55.341 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.336+0000 7ff899432700 1 -- 172.21.15.40:0/1108075700 wait complete. 2020-11-20T13:56:55.349 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T13:56:08.330017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T13:56:17.939684+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T13:56:20.563740+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T13:56:23.062623+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T13:56:55.350 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-20T13:56:55.471 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.465+0000 7f66a1b2b700 1 Processor -- start 2020-11-20T13:56:55.472 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.465+0000 7f66a1b2b700 1 -- start start 2020-11-20T13:56:55.472 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.466+0000 7f66a1b2b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f669c05fa50 0x7f669c0b3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:55.472 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.466+0000 7f66a1b2b700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f669c0b6900 0x7f669c145290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:55.473 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.466+0000 7f66a1b2b700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f669c1489e0 con 0x7f669c13f400 2020-11-20T13:56:55.477 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.466+0000 7f66a1b2b700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f669c148c80 con 0x7f669c05fa50 2020-11-20T13:56:55.478 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.466+0000 7f66a1b2b700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f669c05f5c0 con 0x7f669c0b6900 2020-11-20T13:56:55.478 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.466+0000 7f669affd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f669c05fa50 0x7f669c0b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:55.478 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.466+0000 7f669affd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f669c05fa50 0x7f669c0b3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55658/0 (socket says 172.21.15.40:55658) 2020-11-20T13:56:55.478 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.466+0000 7f669affd700 1 -- 172.21.15.40:0/2468738131 learned_addr learned my addr 172.21.15.40:0/2468738131 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:55.478 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.466+0000 7f669bfff700 1 --2- 172.21.15.40:0/2468738131 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f669c0b6900 0x7f669c145290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:55.479 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.467+0000 7f669affd700 1 -- 172.21.15.40:0/2468738131 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f669c0b6900 msgr2=0x7f669c145290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:55.479 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.467+0000 7f669affd700 1 --2- 172.21.15.40:0/2468738131 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f669c0b6900 0x7f669c145290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:55.479 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.467+0000 7f669affd700 1 -- 172.21.15.40:0/2468738131 >> v1:172.21.15.116:6789/0 conn(0x7f669c13f400 legacy=0x7f669c149120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:55.479 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.467+0000 7f669affd700 1 -- 172.21.15.40:0/2468738131 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f669c0b6db0 con 0x7f669c05fa50 2020-11-20T13:56:55.480 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.467+0000 7f669affd700 1 --2- 172.21.15.40:0/2468738131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f669c05fa50 0x7f669c0b3730 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f6690002f30 tx=0x7f6690006e80).ready entity=mon.0 client_cookie=8601ec9c38a48e18 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:55.480 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.467+0000 7f669a7fc700 1 -- 172.21.15.40:0/2468738131 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66900097f0 con 0x7f669c05fa50 2020-11-20T13:56:55.480 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.467+0000 7f669a7fc700 1 -- 172.21.15.40:0/2468738131 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6690009950 con 0x7f669c05fa50 2020-11-20T13:56:55.480 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.467+0000 7f669a7fc700 1 -- 172.21.15.40:0/2468738131 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6690009cb0 con 0x7f669c05fa50 2020-11-20T13:56:55.480 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.467+0000 7f66a1b2b700 1 -- 172.21.15.40:0/2468738131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f669c05fa50 msgr2=0x7f669c0b3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:55.481 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.467+0000 7f66a1b2b700 1 --2- 172.21.15.40:0/2468738131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f669c05fa50 0x7f669c0b3730 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f6690002f30 tx=0x7f6690006e80).stop 2020-11-20T13:56:55.482 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.467+0000 7f66a1b2b700 1 -- 172.21.15.40:0/2468738131 shutdown_connections 2020-11-20T13:56:55.482 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.467+0000 7f66a1b2b700 1 --2- 172.21.15.40:0/2468738131 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f669c0b6900 0x7f669c145290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:55.482 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.467+0000 7f66a1b2b700 1 --2- 172.21.15.40:0/2468738131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f669c05fa50 0x7f669c0b3730 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:55.483 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.467+0000 7f66a1b2b700 1 -- 172.21.15.40:0/2468738131 >> 172.21.15.40:0/2468738131 conn(0x7f669c0ba1d0 msgr2=0x7f669c0b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:55.484 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.467+0000 7f66a1b2b700 1 -- 172.21.15.40:0/2468738131 shutdown_connections 2020-11-20T13:56:55.484 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.467+0000 7f66a1b2b700 1 -- 172.21.15.40:0/2468738131 wait complete. 2020-11-20T13:56:55.484 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.468+0000 7f66a1b2b700 1 Processor -- start 2020-11-20T13:56:55.485 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.469+0000 7f66a1b2b700 1 -- start start 2020-11-20T13:56:55.485 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.469+0000 7f66a1b2b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f669c05fa50 0x7f669c0676c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:55.485 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.469+0000 7f66a1b2b700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f669c0b6900 0x7f669c1571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:55.485 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.469+0000 7f66a1b2b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f669c13f400 0x7f669c12ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:55.485 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.469+0000 7f66a1b2b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669c067bb0 con 0x7f669c05fa50 2020-11-20T13:56:55.486 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.469+0000 7f66a1b2b700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669c1576d0 con 0x7f669c13f400 2020-11-20T13:56:55.487 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.469+0000 7f66a1b2b700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f669c156800 con 0x7f669c0b6900 2020-11-20T13:56:55.488 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.469+0000 7f669b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f669c05fa50 0x7f669c0676c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:55.495 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.469+0000 7f669b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f669c05fa50 0x7f669c0676c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55664/0 (socket says 172.21.15.40:55664) 2020-11-20T13:56:55.495 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.469+0000 7f669b7fe700 1 -- 172.21.15.40:0/667748523 learned_addr learned my addr 172.21.15.40:0/667748523 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:55.496 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.469+0000 7f669affd700 1 --2- 172.21.15.40:0/667748523 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f669c0b6900 0x7f669c1571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:55.496 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.469+0000 7f669bfff700 1 --2- 172.21.15.40:0/667748523 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f669c13f400 0x7f669c12ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:55.496 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.470+0000 7f669b7fe700 1 -- 172.21.15.40:0/667748523 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f669c0b6900 msgr2=0x7f669c1571e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:55.496 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.470+0000 7f669b7fe700 1 --2- 172.21.15.40:0/667748523 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f669c0b6900 0x7f669c1571e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:55.497 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.470+0000 7f669b7fe700 1 -- 172.21.15.40:0/667748523 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f669c13f400 msgr2=0x7f669c12ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:55.497 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.470+0000 7f669b7fe700 1 --2- 172.21.15.40:0/667748523 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f669c13f400 0x7f669c12ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:55.497 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.470+0000 7f669b7fe700 1 -- 172.21.15.40:0/667748523 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f669c156aa0 con 0x7f669c05fa50 2020-11-20T13:56:55.497 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.470+0000 7f669bfff700 1 --2- 172.21.15.40:0/667748523 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f669c13f400 0x7f669c12ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:55.497 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.470+0000 7f669affd700 1 --2- 172.21.15.40:0/667748523 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f669c0b6900 0x7f669c1571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:55.498 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.470+0000 7f669b7fe700 1 --2- 172.21.15.40:0/667748523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f669c05fa50 0x7f669c0676c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f669c157860 tx=0x7f668c007b40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:55.498 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.470+0000 7f6698ff9700 1 -- 172.21.15.40:0/667748523 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f668c00e4a0 con 0x7f669c05fa50 2020-11-20T13:56:55.498 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.470+0000 7f6698ff9700 1 -- 172.21.15.40:0/667748523 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f668c00e600 con 0x7f669c05fa50 2020-11-20T13:56:55.498 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.470+0000 7f66a1b2b700 1 -- 172.21.15.40:0/667748523 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f669c156d40 con 0x7f669c05fa50 2020-11-20T13:56:55.498 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.470+0000 7f6698ff9700 1 -- 172.21.15.40:0/667748523 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f668c00e960 con 0x7f669c05fa50 2020-11-20T13:56:55.499 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.470+0000 7f66a1b2b700 1 -- 172.21.15.40:0/667748523 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f669c0b2a40 con 0x7f669c05fa50 2020-11-20T13:56:55.500 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.471+0000 7f66a1b2b700 1 -- 172.21.15.40:0/667748523 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f669c057050 con 0x7f669c05fa50 2020-11-20T13:56:55.500 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.471+0000 7f6698ff9700 1 -- 172.21.15.40:0/667748523 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f668c010070 con 0x7f669c05fa50 2020-11-20T13:56:55.500 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.471+0000 7f6698ff9700 1 --2- 172.21.15.40:0/667748523 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f667c051290 0x7f667c053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:55.501 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.471+0000 7f6698ff9700 1 -- 172.21.15.40:0/667748523 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f668c068520 con 0x7f669c05fa50 2020-11-20T13:56:55.506 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.472+0000 7f669affd700 1 --2- 172.21.15.40:0/667748523 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f667c051290 0x7f667c053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:55.506 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.472+0000 7f669affd700 1 --2- 172.21.15.40:0/667748523 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f667c051290 0x7f667c053750 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f669001a000 tx=0x7f6690015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:55.506 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.475+0000 7f6698ff9700 1 -- 172.21.15.40:0/667748523 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f668c040740 con 0x7f669c05fa50 2020-11-20T13:56:55.636 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.632+0000 7f66a1b2b700 1 -- 172.21.15.40:0/667748523 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f669c0b5fd0 con 0x7f669c05fa50 2020-11-20T13:56:55.636 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.632+0000 7f6698ff9700 1 -- 172.21.15.40:0/667748523 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v24) v1 ==== 110+0+10 (secure 0 0 0) 0x7f668c007c60 con 0x7f669c05fa50 2020-11-20T13:56:55.636 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.633+0000 7f66a1b2b700 1 -- 172.21.15.40:0/667748523 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f667c051290 msgr2=0x7f667c053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:55.636 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.633+0000 7f66a1b2b700 1 --2- 172.21.15.40:0/667748523 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f667c051290 0x7f667c053750 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f669001a000 tx=0x7f6690015000).stop 2020-11-20T13:56:55.637 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.633+0000 7f66a1b2b700 1 -- 172.21.15.40:0/667748523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f669c05fa50 msgr2=0x7f669c0676c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:55.637 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.633+0000 7f66a1b2b700 1 --2- 172.21.15.40:0/667748523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f669c05fa50 0x7f669c0676c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f669c157860 tx=0x7f668c007b40).stop 2020-11-20T13:56:55.637 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.633+0000 7f66a1b2b700 1 -- 172.21.15.40:0/667748523 shutdown_connections 2020-11-20T13:56:55.637 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.633+0000 7f66a1b2b700 1 --2- 172.21.15.40:0/667748523 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f667c051290 0x7f667c053750 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:55.637 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.633+0000 7f66a1b2b700 1 --2- 172.21.15.40:0/667748523 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f669c13f400 0x7f669c12ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:55.638 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.633+0000 7f66a1b2b700 1 --2- 172.21.15.40:0/667748523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f669c05fa50 0x7f669c0676c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:55.638 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.633+0000 7f66a1b2b700 1 --2- 172.21.15.40:0/667748523 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f669c0b6900 0x7f669c1571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:55.638 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.633+0000 7f66a1b2b700 1 -- 172.21.15.40:0/667748523 >> 172.21.15.40:0/667748523 conn(0x7f669c0ba1d0 msgr2=0x7f669c0b7650 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:55.638 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.633+0000 7f66a1b2b700 1 -- 172.21.15.40:0/667748523 shutdown_connections 2020-11-20T13:56:55.639 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.633+0000 7f66a1b2b700 1 -- 172.21.15.40:0/667748523 wait complete. 2020-11-20T13:56:55.641 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2020-11-20T13:56:55.645 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-20T13:56:55.766 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.762+0000 7fc01494b700 1 Processor -- start 2020-11-20T13:56:55.767 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.762+0000 7fc01494b700 1 -- start start 2020-11-20T13:56:55.767 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.763+0000 7fc01494b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc0100b6c80 0x7fc0100b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:55.768 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.763+0000 7fc01494b700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc010144bd0 0x7fc0100b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:55.768 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.763+0000 7fc01494b700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc0100b45e0 0x7fc0100b1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:55.768 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.763+0000 7fc01494b700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc010145030 con 0x7fc0100b6c80 2020-11-20T13:56:55.768 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.763+0000 7fc01494b700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0100b0d10 con 0x7fc0100b45e0 2020-11-20T13:56:55.768 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.763+0000 7fc01494b700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0100b0fb0 con 0x7fc010144bd0 2020-11-20T13:56:55.769 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.763+0000 7fc00e59c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc0100b6c80 0x7fc0100b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:55.769 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.763+0000 7fc00e59c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc0100b6c80 0x7fc0100b31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55672/0 (socket says 172.21.15.40:55672) 2020-11-20T13:56:55.769 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.763+0000 7fc00e59c700 1 -- 172.21.15.40:0/1356721714 learned_addr learned my addr 172.21.15.40:0/1356721714 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:55.769 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.763+0000 7fc00ed9d700 1 --2- 172.21.15.40:0/1356721714 >> v2:172.21.15.116:3300/0 conn(0x7fc0100b45e0 0x7fc0100b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:55.770 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.763+0000 7fc00dd9b700 1 --2- 172.21.15.40:0/1356721714 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc010144bd0 0x7fc0100b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:55.770 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.764+0000 7fc00e59c700 1 -- 172.21.15.40:0/1356721714 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc010144bd0 msgr2=0x7fc0100b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:55.770 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.764+0000 7fc00e59c700 1 --2- 172.21.15.40:0/1356721714 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc010144bd0 0x7fc0100b40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:55.770 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.764+0000 7fc00e59c700 1 -- 172.21.15.40:0/1356721714 >> v2:172.21.15.116:3300/0 conn(0x7fc0100b45e0 msgr2=0x7fc0100b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:55.770 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.764+0000 7fc00e59c700 1 --2- 172.21.15.40:0/1356721714 >> v2:172.21.15.116:3300/0 conn(0x7fc0100b45e0 0x7fc0100b1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:55.771 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.764+0000 7fc00e59c700 1 -- 172.21.15.40:0/1356721714 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0100b7060 con 0x7fc0100b6c80 2020-11-20T13:56:55.771 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.764+0000 7fc00ed9d700 1 --2- 172.21.15.40:0/1356721714 >> v2:172.21.15.116:3300/0 conn(0x7fc0100b45e0 0x7fc0100b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:55.771 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.764+0000 7fc00e59c700 1 --2- 172.21.15.40:0/1356721714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0100b6c80 0x7fc0100b31f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7fc004002f30 tx=0x7fc004006e20).ready entity=mon.0 client_cookie=d1394d2bcd1139d2 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:55.771 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.764+0000 7fc00d59a700 1 -- 172.21.15.40:0/1356721714 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc0040097f0 con 0x7fc0100b6c80 2020-11-20T13:56:55.772 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.764+0000 7fc00d59a700 1 -- 172.21.15.40:0/1356721714 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc004009950 con 0x7fc0100b6c80 2020-11-20T13:56:55.772 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.764+0000 7fc00d59a700 1 -- 172.21.15.40:0/1356721714 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc004009cb0 con 0x7fc0100b6c80 2020-11-20T13:56:55.772 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.764+0000 7fc01494b700 1 -- 172.21.15.40:0/1356721714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0100b6c80 msgr2=0x7fc0100b31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:55.773 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.764+0000 7fc01494b700 1 --2- 172.21.15.40:0/1356721714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0100b6c80 0x7fc0100b31f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7fc004002f30 tx=0x7fc004006e20).stop 2020-11-20T13:56:55.773 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.764+0000 7fc01494b700 1 -- 172.21.15.40:0/1356721714 shutdown_connections 2020-11-20T13:56:55.773 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.764+0000 7fc01494b700 1 --2- 172.21.15.40:0/1356721714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0100b6c80 0x7fc0100b31f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:55.774 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.765+0000 7fc01494b700 1 --2- 172.21.15.40:0/1356721714 >> v2:172.21.15.116:3300/0 conn(0x7fc0100b45e0 0x7fc0100b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:55.774 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.765+0000 7fc01494b700 1 --2- 172.21.15.40:0/1356721714 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc010144bd0 0x7fc0100b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:55.774 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.765+0000 7fc01494b700 1 -- 172.21.15.40:0/1356721714 >> 172.21.15.40:0/1356721714 conn(0x7fc0100ba1d0 msgr2=0x7fc0100b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:55.774 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.765+0000 7fc01494b700 1 -- 172.21.15.40:0/1356721714 shutdown_connections 2020-11-20T13:56:55.775 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.765+0000 7fc01494b700 1 -- 172.21.15.40:0/1356721714 wait complete. 2020-11-20T13:56:55.775 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.766+0000 7fc01494b700 1 Processor -- start 2020-11-20T13:56:55.775 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.766+0000 7fc01494b700 1 -- start start 2020-11-20T13:56:55.778 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.766+0000 7fc01494b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0100b45e0 0x7fc010067000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:55.778 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.766+0000 7fc01494b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0100b6c80 0x7fc0101430f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:55.778 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.766+0000 7fc01494b700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc010144bd0 0x7fc01012e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:55.779 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.766+0000 7fc01494b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc010143c30 con 0x7fc0100b6c80 2020-11-20T13:56:55.779 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.766+0000 7fc01494b700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc010143990 con 0x7fc0100b45e0 2020-11-20T13:56:55.779 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.766+0000 7fc01494b700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc010156590 con 0x7fc010144bd0 2020-11-20T13:56:55.779 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.766+0000 7fc00dd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0100b6c80 0x7fc0101430f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:55.779 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.766+0000 7fc00dd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0100b6c80 0x7fc0101430f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55680/0 (socket says 172.21.15.40:55680) 2020-11-20T13:56:55.780 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.766+0000 7fc00dd9b700 1 -- 172.21.15.40:0/1878068194 learned_addr learned my addr 172.21.15.40:0/1878068194 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:55.780 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.767+0000 7fc00e59c700 1 --2- 172.21.15.40:0/1878068194 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0100b45e0 0x7fc010067000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:55.780 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.767+0000 7fc00ed9d700 1 --2- 172.21.15.40:0/1878068194 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc010144bd0 0x7fc01012e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:55.780 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.767+0000 7fc00dd9b700 1 -- 172.21.15.40:0/1878068194 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc010144bd0 msgr2=0x7fc01012e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:55.780 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.767+0000 7fc00dd9b700 1 --2- 172.21.15.40:0/1878068194 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc010144bd0 0x7fc01012e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:55.781 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.767+0000 7fc00dd9b700 1 -- 172.21.15.40:0/1878068194 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0100b45e0 msgr2=0x7fc010067000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:55.781 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.767+0000 7fc00dd9b700 1 --2- 172.21.15.40:0/1878068194 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0100b45e0 0x7fc010067000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:55.781 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.767+0000 7fc00dd9b700 1 -- 172.21.15.40:0/1878068194 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0101435e0 con 0x7fc0100b6c80 2020-11-20T13:56:55.782 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.767+0000 7fc00ed9d700 1 --2- 172.21.15.40:0/1878068194 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc010144bd0 0x7fc01012e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:55.782 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.767+0000 7fc00dd9b700 1 --2- 172.21.15.40:0/1878068194 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0100b6c80 0x7fc0101430f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7fc00000c000 tx=0x7fc000006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:55.782 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.767+0000 7fbfff7fe700 1 -- 172.21.15.40:0/1878068194 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc00000ca80 con 0x7fc0100b6c80 2020-11-20T13:56:55.783 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.767+0000 7fbfff7fe700 1 -- 172.21.15.40:0/1878068194 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc00000cbe0 con 0x7fc0100b6c80 2020-11-20T13:56:55.783 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.767+0000 7fc01494b700 1 -- 172.21.15.40:0/1878068194 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc01015a670 con 0x7fc0100b6c80 2020-11-20T13:56:55.783 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.767+0000 7fbfff7fe700 1 -- 172.21.15.40:0/1878068194 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc0000123f0 con 0x7fc0100b6c80 2020-11-20T13:56:55.783 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.767+0000 7fc01494b700 1 -- 172.21.15.40:0/1878068194 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc01012ee90 con 0x7fc0100b6c80 2020-11-20T13:56:55.784 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.768+0000 7fc01494b700 1 -- 172.21.15.40:0/1878068194 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc010156750 con 0x7fc0100b6c80 2020-11-20T13:56:55.784 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.768+0000 7fbfff7fe700 1 -- 172.21.15.40:0/1878068194 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fc000012550 con 0x7fc0100b6c80 2020-11-20T13:56:55.784 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.768+0000 7fbfff7fe700 1 --2- 172.21.15.40:0/1878068194 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fbfec051320 0x7fbfec0537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:55.784 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.768+0000 7fbfff7fe700 1 -- 172.21.15.40:0/1878068194 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fc000069680 con 0x7fc0100b6c80 2020-11-20T13:56:55.784 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.769+0000 7fc00e59c700 1 --2- 172.21.15.40:0/1878068194 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fbfec051320 0x7fbfec0537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:55.785 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.769+0000 7fc00e59c700 1 --2- 172.21.15.40:0/1878068194 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fbfec051320 0x7fbfec0537e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7fc004019000 tx=0x7fc004015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:55.785 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.772+0000 7fbfff7fe700 1 -- 172.21.15.40:0/1878068194 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fc000041730 con 0x7fc0100b6c80 2020-11-20T13:56:55.936 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.931+0000 7fc01494b700 1 -- 172.21.15.40:0/1878068194 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc010057050 con 0x7fc0100b6c80 2020-11-20T13:56:55.936 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.932+0000 7fbfff7fe700 1 -- 172.21.15.40:0/1878068194 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc000044d50 con 0x7fc0100b6c80 2020-11-20T13:56:55.936 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.933+0000 7fc01494b700 1 -- 172.21.15.40:0/1878068194 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fbfec051320 msgr2=0x7fbfec0537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:55.936 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.933+0000 7fc01494b700 1 --2- 172.21.15.40:0/1878068194 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fbfec051320 0x7fbfec0537e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7fc004019000 tx=0x7fc004015000).stop 2020-11-20T13:56:55.937 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.933+0000 7fc01494b700 1 -- 172.21.15.40:0/1878068194 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0100b6c80 msgr2=0x7fc0101430f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:55.937 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.933+0000 7fc01494b700 1 --2- 172.21.15.40:0/1878068194 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0100b6c80 0x7fc0101430f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7fc00000c000 tx=0x7fc000006dc0).stop 2020-11-20T13:56:55.938 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.933+0000 7fc01494b700 1 -- 172.21.15.40:0/1878068194 shutdown_connections 2020-11-20T13:56:55.938 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.933+0000 7fc01494b700 1 --2- 172.21.15.40:0/1878068194 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7fbfec051320 0x7fbfec0537e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:55.938 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.933+0000 7fc01494b700 1 --2- 172.21.15.40:0/1878068194 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc010144bd0 0x7fc01012e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:55.938 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.933+0000 7fc01494b700 1 --2- 172.21.15.40:0/1878068194 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0100b45e0 0x7fc010067000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:55.939 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.933+0000 7fc01494b700 1 --2- 172.21.15.40:0/1878068194 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0100b6c80 0x7fc0101430f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:55.939 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.933+0000 7fc01494b700 1 -- 172.21.15.40:0/1878068194 >> 172.21.15.40:0/1878068194 conn(0x7fc0100ba1d0 msgr2=0x7fc0100b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:55.939 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.933+0000 7fc01494b700 1 -- 172.21.15.40:0/1878068194 shutdown_connections 2020-11-20T13:56:55.939 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:55.933+0000 7fc01494b700 1 -- 172.21.15.40:0/1878068194 wait complete. 2020-11-20T13:56:55.940 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2020-11-20T13:56:55.949 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-20T13:56:56.065 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.064+0000 7f1ee55f5700 1 Processor -- start 2020-11-20T13:56:56.066 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.064+0000 7f1ee55f5700 1 -- start start 2020-11-20T13:56:56.073 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.065+0000 7f1ee55f5700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1ee00b31f0 0x7f1ee0144bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:56.073 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.065+0000 7f1ee55f5700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1ee00b1450 0x7f1ee00b1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:56.077 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.065+0000 7f1ee55f5700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ee00b1df0 con 0x7f1ee00b6c80 2020-11-20T13:56:56.077 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.065+0000 7f1ee55f5700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ee00b0d10 con 0x7f1ee00b1450 2020-11-20T13:56:56.077 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.065+0000 7f1ee55f5700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee00b0fb0 con 0x7f1ee00b31f0 2020-11-20T13:56:56.077 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.065+0000 7f1edf7fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1ee00b1450 0x7f1ee00b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:56.078 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.065+0000 7f1edf7fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1ee00b1450 0x7f1ee00b1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55690/0 (socket says 172.21.15.40:55690) 2020-11-20T13:56:56.078 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.065+0000 7f1edf7fe700 1 -- 172.21.15.40:0/4021655829 learned_addr learned my addr 172.21.15.40:0/4021655829 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:56.078 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.065+0000 7f1ede7fc700 1 --2- 172.21.15.40:0/4021655829 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1ee00b31f0 0x7f1ee0144bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:56.078 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.066+0000 7f1edf7fe700 1 -- 172.21.15.40:0/4021655829 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1ee00b31f0 msgr2=0x7f1ee0144bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:56.080 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.066+0000 7f1edf7fe700 1 --2- 172.21.15.40:0/4021655829 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1ee00b31f0 0x7f1ee0144bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:56.080 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.066+0000 7f1edf7fe700 1 -- 172.21.15.40:0/4021655829 >> v1:172.21.15.116:6789/0 conn(0x7f1ee00b6c80 legacy=0x7f1ee00b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:56.080 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.066+0000 7f1edf7fe700 1 -- 172.21.15.40:0/4021655829 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee00b7060 con 0x7f1ee00b1450 2020-11-20T13:56:56.081 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.066+0000 7f1edf7fe700 1 --2- 172.21.15.40:0/4021655829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ee00b1450 0x7f1ee00b1900 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f1ee00b3100 tx=0x7f1ed8009270).ready entity=mon.0 client_cookie=33ae8c96c266a0f6 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:56.081 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.066+0000 7f1eddffb700 1 -- 172.21.15.40:0/4021655829 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ed8009e90 con 0x7f1ee00b1450 2020-11-20T13:56:56.081 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.066+0000 7f1eddffb700 1 -- 172.21.15.40:0/4021655829 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed800bbc0 con 0x7f1ee00b1450 2020-11-20T13:56:56.081 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.066+0000 7f1eddffb700 1 -- 172.21.15.40:0/4021655829 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ed800bd30 con 0x7f1ee00b1450 2020-11-20T13:56:56.082 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.066+0000 7f1ee55f5700 1 -- 172.21.15.40:0/4021655829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ee00b1450 msgr2=0x7f1ee00b1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:56.082 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.066+0000 7f1ee55f5700 1 --2- 172.21.15.40:0/4021655829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ee00b1450 0x7f1ee00b1900 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f1ee00b3100 tx=0x7f1ed8009270).stop 2020-11-20T13:56:56.082 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.067+0000 7f1ee55f5700 1 -- 172.21.15.40:0/4021655829 shutdown_connections 2020-11-20T13:56:56.082 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.067+0000 7f1ee55f5700 1 --2- 172.21.15.40:0/4021655829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ee00b1450 0x7f1ee00b1900 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:56.082 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.067+0000 7f1ee55f5700 1 --2- 172.21.15.40:0/4021655829 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1ee00b31f0 0x7f1ee0144bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:56.083 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.067+0000 7f1ee55f5700 1 -- 172.21.15.40:0/4021655829 >> 172.21.15.40:0/4021655829 conn(0x7f1ee00ba1d0 msgr2=0x7f1ee00b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:56.083 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.067+0000 7f1ee55f5700 1 -- 172.21.15.40:0/4021655829 shutdown_connections 2020-11-20T13:56:56.083 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.067+0000 7f1ee55f5700 1 -- 172.21.15.40:0/4021655829 wait complete. 2020-11-20T13:56:56.084 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.068+0000 7f1ee55f5700 1 Processor -- start 2020-11-20T13:56:56.084 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.068+0000 7f1ee55f5700 1 -- start start 2020-11-20T13:56:56.084 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.068+0000 7f1ee55f5700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1ee00b1450 0x7f1ee0064840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:56.085 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.068+0000 7f1ee55f5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ee00b31f0 0x7f1ee00aeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:56.085 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.068+0000 7f1ee55f5700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ee00b6c80 0x7f1ee00af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:56.085 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.068+0000 7f1ee55f5700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee0063240 con 0x7f1ee00b31f0 2020-11-20T13:56:56.085 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.068+0000 7f1ee55f5700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee00634b0 con 0x7f1ee00b6c80 2020-11-20T13:56:56.086 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.068+0000 7f1ee55f5700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee0063bf0 con 0x7f1ee00b1450 2020-11-20T13:56:56.086 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.068+0000 7f1ede7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ee00b31f0 0x7f1ee00aeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:56.086 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.068+0000 7f1ede7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ee00b31f0 0x7f1ee00aeeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55694/0 (socket says 172.21.15.40:55694) 2020-11-20T13:56:56.086 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.068+0000 7f1ede7fc700 1 -- 172.21.15.40:0/3332056847 learned_addr learned my addr 172.21.15.40:0/3332056847 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:56.087 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.069+0000 7f1edeffd700 1 --2- 172.21.15.40:0/3332056847 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1ee00b1450 0x7f1ee0064840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:56.087 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.069+0000 7f1edf7fe700 1 --2- 172.21.15.40:0/3332056847 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ee00b6c80 0x7f1ee00af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:56.087 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.069+0000 7f1ede7fc700 1 -- 172.21.15.40:0/3332056847 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1ee00b1450 msgr2=0x7f1ee0064840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:56.087 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.069+0000 7f1ede7fc700 1 --2- 172.21.15.40:0/3332056847 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1ee00b1450 0x7f1ee0064840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:56.087 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.069+0000 7f1ede7fc700 1 -- 172.21.15.40:0/3332056847 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ee00b6c80 msgr2=0x7f1ee00af3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:56.088 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.069+0000 7f1ede7fc700 1 --2- 172.21.15.40:0/3332056847 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ee00b6c80 0x7f1ee00af3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:56.088 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.069+0000 7f1ede7fc700 1 -- 172.21.15.40:0/3332056847 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee0063db0 con 0x7f1ee00b31f0 2020-11-20T13:56:56.088 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.069+0000 7f1ede7fc700 1 --2- 172.21.15.40:0/3332056847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ee00b31f0 0x7f1ee00aeeb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f1ed000c000 tx=0x7f1ed0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:56.088 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.069+0000 7f1ecffff700 1 -- 172.21.15.40:0/3332056847 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ed000ca10 con 0x7f1ee00b31f0 2020-11-20T13:56:56.088 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.069+0000 7f1ecffff700 1 -- 172.21.15.40:0/3332056847 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed000cb70 con 0x7f1ee00b31f0 2020-11-20T13:56:56.089 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.069+0000 7f1ee55f5700 1 -- 172.21.15.40:0/3332056847 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ee0156910 con 0x7f1ee00b31f0 2020-11-20T13:56:56.089 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.069+0000 7f1ecffff700 1 -- 172.21.15.40:0/3332056847 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ed001a470 con 0x7f1ee00b31f0 2020-11-20T13:56:56.090 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.069+0000 7f1ee55f5700 1 -- 172.21.15.40:0/3332056847 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ee0156e00 con 0x7f1ee00b31f0 2020-11-20T13:56:56.091 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.070+0000 7f1ecffff700 1 -- 172.21.15.40:0/3332056847 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f1ed001a5d0 con 0x7f1ee00b31f0 2020-11-20T13:56:56.091 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.070+0000 7f1ee55f5700 1 -- 172.21.15.40:0/3332056847 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ee0057050 con 0x7f1ee00b31f0 2020-11-20T13:56:56.091 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.070+0000 7f1ecffff700 1 --2- 172.21.15.40:0/3332056847 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f1ebc051320 0x7f1ebc0537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:56.092 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.070+0000 7f1ecffff700 1 -- 172.21.15.40:0/3332056847 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f1ed006b4e0 con 0x7f1ee00b31f0 2020-11-20T13:56:56.092 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.071+0000 7f1edeffd700 1 --2- 172.21.15.40:0/3332056847 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f1ebc051320 0x7f1ebc0537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:56.092 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.071+0000 7f1edeffd700 1 --2- 172.21.15.40:0/3332056847 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f1ebc051320 0x7f1ebc0537e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f1ed400d000 tx=0x7f1ed4007ae0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:56.093 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.074+0000 7f1ecffff700 1 -- 172.21.15.40:0/3332056847 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f1ed00436a0 con 0x7f1ee00b31f0 2020-11-20T13:56:56.234 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.233+0000 7f1ee55f5700 1 -- 172.21.15.40:0/3332056847 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1ee005edb0 con 0x7f1ee00b31f0 2020-11-20T13:56:56.235 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.233+0000 7f1ecffff700 1 -- 172.21.15.40:0/3332056847 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+10 (secure 0 0 0) 0x7f1ed0046cc0 con 0x7f1ee00b31f0 2020-11-20T13:56:56.235 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2020-11-20T13:56:56.236 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.235+0000 7f1ee55f5700 1 -- 172.21.15.40:0/3332056847 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f1ebc051320 msgr2=0x7f1ebc0537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:56.237 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.235+0000 7f1ee55f5700 1 --2- 172.21.15.40:0/3332056847 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f1ebc051320 0x7f1ebc0537e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f1ed400d000 tx=0x7f1ed4007ae0).stop 2020-11-20T13:56:56.237 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.235+0000 7f1ee55f5700 1 -- 172.21.15.40:0/3332056847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ee00b31f0 msgr2=0x7f1ee00aeeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:56.237 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.235+0000 7f1ee55f5700 1 --2- 172.21.15.40:0/3332056847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ee00b31f0 0x7f1ee00aeeb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f1ed000c000 tx=0x7f1ed0008040).stop 2020-11-20T13:56:56.237 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.235+0000 7f1ee55f5700 1 -- 172.21.15.40:0/3332056847 shutdown_connections 2020-11-20T13:56:56.238 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.235+0000 7f1ee55f5700 1 --2- 172.21.15.40:0/3332056847 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f1ebc051320 0x7f1ebc0537e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:56.238 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.235+0000 7f1ee55f5700 1 --2- 172.21.15.40:0/3332056847 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1ee00b1450 0x7f1ee0064840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:56.238 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.235+0000 7f1ee55f5700 1 --2- 172.21.15.40:0/3332056847 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ee00b6c80 0x7f1ee00af3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:56.238 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.235+0000 7f1ee55f5700 1 --2- 172.21.15.40:0/3332056847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ee00b31f0 0x7f1ee00aeeb0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:56.239 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.235+0000 7f1ee55f5700 1 -- 172.21.15.40:0/3332056847 >> 172.21.15.40:0/3332056847 conn(0x7f1ee00ba1d0 msgr2=0x7f1ee00b84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:56.239 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.235+0000 7f1ee55f5700 1 -- 172.21.15.40:0/3332056847 shutdown_connections 2020-11-20T13:56:56.239 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.235+0000 7f1ee55f5700 1 -- 172.21.15.40:0/3332056847 wait complete. 2020-11-20T13:56:56.246 INFO:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-20T13:56:56.369 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.362+0000 7f6bf9306700 1 Processor -- start 2020-11-20T13:56:56.369 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.362+0000 7f6bf9306700 1 -- start start 2020-11-20T13:56:56.369 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.363+0000 7f6bf9306700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6bf4144390 0x7f6bf414df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:56.369 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.363+0000 7f6bf9306700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bf40b5fd0 con 0x7f6bf40b3960 2020-11-20T13:56:56.371 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.363+0000 7f6bf9306700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bf40b05f0 con 0x7f6bf40b5bf0 2020-11-20T13:56:56.371 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.363+0000 7f6bf9306700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bf4144870 con 0x7f6bf4144390 2020-11-20T13:56:56.371 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.363+0000 7f6bf37fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6bf4144390 0x7f6bf414df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:56.371 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.363+0000 7f6bf37fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6bf4144390 0x7f6bf414df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55700/0 (socket says 172.21.15.40:55700) 2020-11-20T13:56:56.371 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.363+0000 7f6bf37fe700 1 -- 172.21.15.40:0/970879902 learned_addr learned my addr 172.21.15.40:0/970879902 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:56.372 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.363+0000 7f6bf27fc700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f6bf40b3960 0x7f6bf405faa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46296/0 (socket says 172.21.15.40:46296) 2020-11-20T13:56:56.372 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.363+0000 7f6bf37fe700 1 -- 172.21.15.40:0/970879902 >> v1:172.21.15.116:6789/0 conn(0x7f6bf40b5bf0 legacy=0x7f6bf405f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:56.372 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.363+0000 7f6bf37fe700 1 -- 172.21.15.40:0/970879902 >> v1:172.21.15.40:6789/0 conn(0x7f6bf40b3960 legacy=0x7f6bf405faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:56.373 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.363+0000 7f6bf37fe700 1 -- 172.21.15.40:0/970879902 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf40b0c30 con 0x7f6bf4144390 2020-11-20T13:56:56.373 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.363+0000 7f6bf1ffb700 1 -- 172.21.15.40:0/970879902 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3001178959 0 0) 0x7f6bf40b5fd0 con 0x7f6bf40b3960 2020-11-20T13:56:56.373 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.363+0000 7f6bf37fe700 1 --2- 172.21.15.40:0/970879902 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bf4144390 0x7f6bf414df60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f6bf40b71f0 tx=0x7f6bec006e80).ready entity=mon.0 client_cookie=be09744e45429a50 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:56.373 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.364+0000 7f6bf1ffb700 1 -- 172.21.15.40:0/970879902 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6bec0097b0 con 0x7f6bf4144390 2020-11-20T13:56:56.374 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.364+0000 7f6bf1ffb700 1 -- 172.21.15.40:0/970879902 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bec009910 con 0x7f6bf4144390 2020-11-20T13:56:56.374 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.364+0000 7f6bf1ffb700 1 -- 172.21.15.40:0/970879902 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6bec009a80 con 0x7f6bf4144390 2020-11-20T13:56:56.374 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.364+0000 7f6bf9306700 1 -- 172.21.15.40:0/970879902 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bf4144390 msgr2=0x7f6bf414df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:56.374 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.364+0000 7f6bf9306700 1 --2- 172.21.15.40:0/970879902 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bf4144390 0x7f6bf414df60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f6bf40b71f0 tx=0x7f6bec006e80).stop 2020-11-20T13:56:56.374 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.364+0000 7f6bf9306700 1 -- 172.21.15.40:0/970879902 shutdown_connections 2020-11-20T13:56:56.375 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.364+0000 7f6bf9306700 1 --2- 172.21.15.40:0/970879902 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bf4144390 0x7f6bf414df60 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:56.375 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.364+0000 7f6bf9306700 1 -- 172.21.15.40:0/970879902 >> 172.21.15.40:0/970879902 conn(0x7f6bf40ba1d0 msgr2=0x7f6bf40b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:56.375 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.364+0000 7f6bf9306700 1 -- 172.21.15.40:0/970879902 shutdown_connections 2020-11-20T13:56:56.375 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.364+0000 7f6bf9306700 1 -- 172.21.15.40:0/970879902 wait complete. 2020-11-20T13:56:56.376 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.365+0000 7f6bf9306700 1 Processor -- start 2020-11-20T13:56:56.376 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.365+0000 7f6bf9306700 1 -- start start 2020-11-20T13:56:56.376 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.366+0000 7f6bf9306700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6bf40b3960 0x7f6bf40aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:56.376 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.366+0000 7f6bf9306700 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6bf40b5bf0 0x7f6bf413c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:56.376 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.366+0000 7f6bf9306700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bf4144390 0x7f6bf413c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:56.377 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.366+0000 7f6bf9306700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf4063f60 con 0x7f6bf4144390 2020-11-20T13:56:56.377 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.366+0000 7f6bf9306700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf4064200 con 0x7f6bf40b3960 2020-11-20T13:56:56.377 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.366+0000 7f6bf9306700 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf40644a0 con 0x7f6bf40b5bf0 2020-11-20T13:56:56.377 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.366+0000 7f6bf37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bf4144390 0x7f6bf413c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:56.379 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.366+0000 7f6bf37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bf4144390 0x7f6bf413c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55710/0 (socket says 172.21.15.40:55710) 2020-11-20T13:56:56.379 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.366+0000 7f6bf37fe700 1 -- 172.21.15.40:0/3928418587 learned_addr learned my addr 172.21.15.40:0/3928418587 (peer_addr_for_me v2:172.21.15.40:0/0) 2020-11-20T13:56:56.379 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.366+0000 7f6bf27fc700 1 --2- 172.21.15.40:0/3928418587 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6bf40b5bf0 0x7f6bf413c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:56.380 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.366+0000 7f6bf2ffd700 1 --2- 172.21.15.40:0/3928418587 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6bf40b3960 0x7f6bf40aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:56.380 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.366+0000 7f6bf37fe700 1 -- 172.21.15.40:0/3928418587 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6bf40b5bf0 msgr2=0x7f6bf413c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:56.382 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.366+0000 7f6bf37fe700 1 --2- 172.21.15.40:0/3928418587 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6bf40b5bf0 0x7f6bf413c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:56.382 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.366+0000 7f6bf37fe700 1 -- 172.21.15.40:0/3928418587 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6bf40b3960 msgr2=0x7f6bf40aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:56.382 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.366+0000 7f6bf37fe700 1 --2- 172.21.15.40:0/3928418587 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6bf40b3960 0x7f6bf40aef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:56.383 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.366+0000 7f6bf37fe700 1 -- 172.21.15.40:0/3928418587 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf4063c50 con 0x7f6bf4144390 2020-11-20T13:56:56.383 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.366+0000 7f6bf2ffd700 1 --2- 172.21.15.40:0/3928418587 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6bf40b3960 0x7f6bf40aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:56.383 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.366+0000 7f6bf37fe700 1 --2- 172.21.15.40:0/3928418587 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bf4144390 0x7f6bf413c7b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f6bf40b71f0 tx=0x7f6bec016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:56.383 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.366+0000 7f6bebfff700 1 -- 172.21.15.40:0/3928418587 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6bec011e70 con 0x7f6bf4144390 2020-11-20T13:56:56.383 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.367+0000 7f6bebfff700 1 -- 172.21.15.40:0/3928418587 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bec020400 con 0x7f6bf4144390 2020-11-20T13:56:56.384 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.367+0000 7f6bebfff700 1 -- 172.21.15.40:0/3928418587 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6bec020760 con 0x7f6bf4144390 2020-11-20T13:56:56.384 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.367+0000 7f6bf9306700 1 -- 172.21.15.40:0/3928418587 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bf413f0a0 con 0x7f6bf4144390 2020-11-20T13:56:56.384 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.367+0000 7f6bf9306700 1 -- 172.21.15.40:0/3928418587 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bf4140cd0 con 0x7f6bf4144390 2020-11-20T13:56:56.384 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.367+0000 7f6bf9306700 1 -- 172.21.15.40:0/3928418587 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bf4064660 con 0x7f6bf4144390 2020-11-20T13:56:56.384 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.368+0000 7f6bebfff700 1 -- 172.21.15.40:0/3928418587 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f6bec01e070 con 0x7f6bf4144390 2020-11-20T13:56:56.385 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.368+0000 7f6bebfff700 1 --2- 172.21.15.40:0/3928418587 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f6bd80513c0 0x7f6bd8053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:56.386 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.368+0000 7f6bebfff700 1 -- 172.21.15.40:0/3928418587 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f6bec069bb0 con 0x7f6bf4144390 2020-11-20T13:56:56.386 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.368+0000 7f6bf2ffd700 1 --2- 172.21.15.40:0/3928418587 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f6bd80513c0 0x7f6bd8053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:56.386 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.368+0000 7f6bf2ffd700 1 --2- 172.21.15.40:0/3928418587 >> [v2:172.21.15.116:6800/22233,v1:172.21.15.116:6801/22233] conn(0x7f6bd80513c0 0x7f6bd8053880 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f6bdc00b000 tx=0x7f6bdc009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:56.386 INFO:teuthology.orchestra.run.smithi040.stderr:2020-11-20T13:56:56.371+0000 7f6bebfff700 1 -- 172.21.15.40:0/3928418587 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f6bec070080 con 0x7f6bf4144390